POSTCaUDOOE 
Montero^,  California 


AD-A283  447 


DT?^ 

1  9,994^ 


V  aK  J^i 


THESIS 


AN  ANALYSIS  OF  DOD  INSPECTOR 
GENERAL’S  STATISTICAL  SAMPLING 
PLAN  FOR  NAVY  REPAIRABLE 
ITEM  PROCUREMENTS 


Thomas  D.  Chase 


June,  1994 


Thesis  Co-Advisors: 


Alan  W.  McMasters 
James  M.  Fremgen 


Approved  for  public  release;  distribution  is  unlimited. 


94-26360 


.A.  X 


94  8 


TR  4® 


459 


REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  No.  0704 


t\iblic  raponiof  burdea  fv  diii  coUactioo  of  iafoniilioB  ■>  BWiinaciif  to  avame  I  boiir  per  mpoaie,  iachidiB(  the  Uqm  for  loviewii  iunctioa,  wrbin 
existing  dju  souices.  gaibcring  and  mainiainng  the  data  needed,  and  coaphiint  and  reviewing  the  coUection  of  infannalian.  Send  rnwanti  rrgwding  this 
burden  estuude  or  any  other  aspect  of  this  ooUection  of  infonaalion,  Bcbsdiag  tuggeshana  for  reduciag  this  burden,  to  WaahingtaB  Hendgnaisrti  Servioea. 
Directorate  for  lafonnation  Operations  and  Reports,  1215  Jefletson  Davis  Highway.  Suite  1204,  Arlingtoa,  VA  22202-4302,  and  to  the  Office  of  Managcaaets 
and  Bu^et,  Paperwoifc  Reductiao  ftoject  (0704-0188)  Washinglan  DC  20503. 


AGENCY  USE  ONLY  (Uave  blank)  2. 


REPORT  DATE 
June  1994 


4.  TITLE  AND  SUBTITLE  AN  ANALYSIS  OF  DOD  INSPECTOR 
GENERAL’S  STATISTICAL  SAMPLING  PLAN  FOR  NAVY 
REPAIRABLE  ITEM  PROCUREMENTS 


6.  AUTHOR(S)  Thomas  D.  Chase 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 
Naval  Postgraduate  School 
Monterey  CA  93943-5000 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


REPORT  TYPE  AND  DATES  COVERED 
Master’s  Thesis 


S.  FUNDING  NUMBERS 


8.  PERFORMING 
ORGANEATTON 
REPORT  NUMBER 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 


12b.  DISTRIBUTION  CODE 
•A 


11.  SUPPLEMENTARY  NOTES  The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect 
the  official  policy  or  position  of  the  Department  of  Defense  or  the  U.S.  GovemmenL 


12a.  DISTRIBUnON/AVAILABIUTY  STATEMENT 

Approved  for  public  release;  distributiem  is  unlimited. 


13.  ABSTRACT  (maximum  200  words) 

A  recent  audit  by  the  Department  of  Defense  Inspector  General  (DODIG)  of  Navy  inventory  control 
points  found  a  high  value  of  purchase  requests  for  repairable  items  that  the  auditors  labeled  as 
unnecessary  or  excessive.  The  doUar  figures  reported  were  based  on  the  auditors’  use  of  stratified 
sampling.  The  thesis  examined  the  auditors’  use  of  stratified  sampling  by  attempting  to  replicate  the 
auditors’  process  of  stratifying  and  sampling.  Uie  author  then  attenq^ted  to  verify  the  auditors’  claimed 
confidence  level  and  precision  of  the  final  result  This  study  questions  the  chosen  sample  size  and 
stratification.  In  addition,  this  thesis  found  that  the  auditors’  actual  precision  was  not  as  tight  as  stated  in 
the  DODIG  audit  report  This  was  caused  by  the  auditors’  emphasis  on  the  very  high  dollar  value  strata 
which  had  only  a  few  purchase  requests  rather  than  on  the  stratum  with  the  largest  number  of  purchase 
requests.  It  was  this  latter  stratum  which  had  the  highest  projected  number  and  total  dollar  value  of 
excesses. 


14.  SUBIECTTERMS  AUDIT;  STRATIFIED  RANDOM  SAMPLING;  CONFIDENCE  LEVELS  15.  NUMBER  OF 

PAGES  81 


SECURITY  CLASSin- 
CATION  OF  REPORT 
Unclassified 


18.  SECURITY  CLASSIH- 
CATION  OF  THIS  PAGE 
Unclassified 


19.  SECURITY  CLASSIH- 
CATION  OF  ABSTRACT 
Unclassified 


16.  PRICE  CODE 


20.  LIMITATION  OF 
ABSTRACT 
UL 


^^roved  Fot  public  release;  distribution  is  unlimited. 

AN  ANALYSIS  OF  DOD  INSPECTOR  GENERAL’S  STAHSTICAL 
SAMPLING  PLAN  FOR  NAVY  REPAIRABLE  ITEM  PROCUREMENTS 

by 

Thomas  D.  Chase 
Lieutenant,  United  States  Navy 
B.S.,  Santa  Clara  University,  1983 

Submitted  in  partial  fulfillment 
of  the  requirements  for  the  degree  of 

MASTER  OF  SCIENCE  IN  MANAGEMENT 

from  the 


Author: 


Approved  by: 


NAVAL  POSTGRADUATE  SCHOOL 
June  1994 

_ ^'^*'*^****-  Q.-  CJ**4\ 

Thomas  D.  Chase 


Department  of  Systems  Management 


11 


ABSTRACT 


A  recent  audit  by  the  Department  of  Defense  Inspector  General  (DODIG)  of  Navy 
inventory  control  points  found  a  high  value  of  imrchase  requests  for  repairable  hems  that 
the  auditors  labeled  as  unnecessary  or  excessive.  The  dollar  figures  rqrorted  were  based 
on  the  auditors’  use  of  stratified  sampling.  This  thesis  examined  the  audhors’  use  of 
stratified  sampling  by  attempting  to  replicate  the  audhors’  process  of  stratifying  and 
sampling.  The  author  then  attempted  to  verify  the  audhors’  claimed  confidence  level  and 
precision  of  the  final  resuh.  This  study  questions  the  chosen  sample  and  sample 
stratification.  In  addition,  this  thesis  found  that  the  audhors’  actual  precision  was  not  as 
tight  as  stated  in  the  DODIG  audit  report.  This  was  caused  by  the  audhors’  emphasis  on 
the  very  high  dollar  value  strata  which  had  only  a  few  purchase  requests  rather  than  on  the 
stratum  with  the  largest  number  of  purchase  requests.  It  was  this  latter  stratum  which  had 
the  highest  projected  number  and  total  dollar  value  of  excesses. 
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L  INTRODUCTION 


A.  MOTIVATION 

This  study  is  based  on  the  results  of  an  economy  and  eflSdency  audit  conducted  by 
the  Department  of  Defoise  Inspector  General  (DODIG)  from  August  1990  throu^ 
November  1991  in  accordance  with  standards  issued  by  the  ComptroUa*  General  of  the 
United  States  (Jones,  Fd}ruary  1, 1993,  p.  2).  The  DODIG  conducts  audits  of  the  Navy's 
inventory  control  points’  (ICP)  procurement  actions  using  a  multi-stage  sampling  plan  that 
incorporates  stratified  sampling.  Using  these  stratified  samples,  DODIG  conducts  audit 
tests  to  determine  which  purchase  actions  are  for  "reasonable"  quantities. 

When  the  results  of  this  audit  were  reported,  they  also  identified  a  total  dollar 
value  of  items  that  were  considered  unnecessary  or  premature  purchases.  In  this  case,  the 
DODIG  identified  potential  monetary  benefits  totaling  S71.7  million,  which  represents  the 
estimated  value  of  unnecessary  purchases.  When  audit  findings  disclose  such  "problem" 
purchases,  obligational  authority  is  reduced  for  the  ICPs.  The  reasoning  is  that,  if  the 
funding  had  not  been  wasted  on  these  particular  procurement  actions,  the  funding  would 
not  have  been  needed  at  all.  In  order  to  avoid  inaccurate  conclusions,  it  is  absolutely 
critical  that  such  audit  findings  accurately  refiect  weaknesses  in  invemory  management. 

B.  OBJECTIVE 

The  focus  of  the  thesis  will  be  to  look  at  how  the  DODIG  incorporates  stratified 
sampling  methodologies  in  auditing  major  procurement  actions  and  then  making  recom¬ 
mendations  to  inventory  control  points.  In  particular,  this  thesis  will  look  at  how  the 
DODIG  gathers  the  audit  results  from  each  stratum  sampled  and  projects  these  findings 
over  the  entire  population  of  procurement  actions.  This  study  explores  the  question  of 
whether  sampling  stratification  with  current  sample  sizes  can  be  used  to  make  projections 
of  potential  monetary  savings  with  a  high  confidence  level  and  tight  precision. 
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C.  RESEARCH  QUESTIONS 

The  primary  question  to  be  answered  in  this  project  is:  "Does  the  DODIG's 
statistical  sampling  plan  and  stratified  sampling  of  procurement  actions  lead  to  the  most 
accurate  conclusions  and  recommendations?"  In  order  to  answer  this  question  compl^dy, 
secondary  questions  concerning  choices  of  stratifications,  size  of  strata,  and  confidence 
levels  within  each  stratum  must  be  addressed.  In  particular,  how  is  tlus  sampling  structure 
used  to  project  errors  in  each  stratum  to  the  population  as  a  whole? 

D.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

The  scope  of  this  study  is  very  narrovdy  defined  in  order  to  remain  focused  on  the 
research  questions  mentioned  above.  In  the  audit  report  analyzed,  there  are  many  areas  of 
debate  that  are  clearly  not  resolved.  Each  one  could  be  the  basis  for  a  separate  study  at  a 
later  date.  This  project  will  look  at  the  baas  for  the  amount  of  potential  monetary 
benefits  suggested  in  the  audit  report.  As  the  report's  Executive  Summary  states,  the 
Navy  disagrees  with  the  basis  for  estimating  potential  monetary  benefits  (Jones,  February 
1,  1993,  p.  u). 

This  project's  primary  focus  is  on  how  the  DODIG  used  sampling  to  make  its 
observations  of  the  ICPs'  procurement  of  repairable  items.  For  example,  this  theas 
considers  the  initial  total  quantity  of  line  items,  the  snudler  universe  of  line  items  chosen  to 
sample,  the  adjusted  universe  of  line  items  to  sample,  the  sample  of  procurement  actions 
to  be  scrutinized  and  the  conclusions  drawn  fi-om  these  samples.  The  area  analyzed 
concerns  the  dollar  value  projection  based  on  these  findings.  In  statistical  terminology, 
this  thesis  primarily  focuses  on  the  dollar  value  estimates  of  the  urmecessary  purchases 
reported  in  the  audit  to  see  if  improvements  could  have  been  made  and,  secondarily, 
examines  the  fi'equency  estimation.  Dollar  value  and  firequency  estimation  are  defined  in 
Chapter  II,  Section  B. 
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This  study  is  not  intended  to  settle  disagreements  on  terminology,  determine 
whether  the  audit  was  poorly  timed,  nor  make  reconunendations  on  how  internal  controls 
should  be  changed.  Instead,  this  thesis  concentrates  on  the  statistical  processes  and  on 
how  they  resulted,  dther  accuratdy  or  inaccuratdy,  in  budgetary  recommendations  for  the 
ICFs  under  review. 

E.  PREVIEW 

The  remaining  chapters  are  organized  in  the  following  manner.  Chi^)ter  II  gives 
background  information  on  stratified  sampling,  estimation  sampling,  confidence  intervals, 
and  the  inventory  management  definition  of  stratification. 

Chapter  HI  presents  all  the  data  provided  by  the  statisticians  at  the  DODIG.  The 
data  is  broken  down  as  described  in  the  audit  report,  and  it  shows  the  conclusions  derived 
firom  this  data.  The  chapter  concludes  with  an  erqrlarution  of  how  the  audit  results  were 
actually  used  by  the  affected  command  when  submitting  a  budget  request 

Ciuq)ter  IV  opens  with  a  discussion  of  how  the  DODIG  made  its  statistical 
decisions.  The  following  section  begins  the  author’s  analyses  of  the  statistical  decisions. 
The  first  aruilysis  is  of  the  selected  universe  size  and  sample  size,  followed  by  an 
alternative  method  to  choose  a  sample  size,  and,  firudly,  how  to  stratify  the  chosen  sample 
size.  The  second  major  analysis  is  of  the  auditors’  stated  confidence  level  and  related 
precision.  The  chapter  closes  with  a  comment  on  the  several  numerical  discrepancies 
found  in  the  audit  report  and  worldng  papers. 

Chapter  V  summarizes  the  main  points  and  conclusions  of  the  thesis  and  presents 
several  recommendations. 
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IL  METHODOLOGY 


A.  STRATIFIED  SAMPLING 

1.  Stnittficd  Sampling  Defined 

In  stratified  random  sampling,  the  population  is  divided  imo  a  smies  of 
independoit  subpopulations  which  are  called  "strata."  Each  stratum  is  sampled  as  though 
it  were  a  separate  population  from  which  an  unrestricted  random  sample  were  being 
drawn.  It  is  then  posable  to  combine  the  separate  results  to  make  a  conclusion  about  the 
entire  population.  A  degree  of  predaon  can  then  be  applied  to  these  results  (Cyert,  1962, 

p.  116). 

2.  Why  stratified  sampling? 

Auditors  may  not  consider  all  accounts  or  records  of  equal  inqx>rtance.  For 
instance,  they  may  have  a  much  greater  interest  in  establishing  the  accuracy  of  large 
accounts  and  be  unwilling  to  run  as  great  a  sampling  risk  for  this  type  of  account. 
Objective  sampling  methods  do  not  necessarily  require  sampling  fix}m  a  general  pool  of 
items.  It  is  not  only  posable  but  often  desirable  to  s^regate  the  population  into  separate 
groups,  by  size  or  other  characteristics,  and  to  sample  vdth  various  degrees  of  accuracy  in 
each  area  separately. 

When  sampling  accounts,  it  is  common  to  want  to  examine  all  of  the  large 
accounts,  a  large  portion  of  the  moderately  azed  accounts,  and  a  relatively  small 
proportion  of  the  lowest  doUar-valued  accounts.  In  order  to  draw  a  clear  concluaon  fix)m 
this  process,  it  is  necessary  to  stratify  the  accounts  by  size  and  then  analyze  the  results. 
This  procedure  is  looked  upon  favorably  because  there  is  improved  sample  reliability  for 
the  large  accounts.  Because  it  removes  the  large  accounts  fi'om  other  sections  of  the 
sample,  determination  of  the  sampling  variability  also  is  more  precise  (Hill,  1962,  p.  45). 
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3.  How  to  Use  Stnitined  Sampling 

With  stratified  sampling  of  audit  units,  the  heterogeneous  population  (Figure 
1  A)  is  first  divided  into  a  nund>er  of  mutually  excluave  groups  or  strata  (I  .B).  After  the 
population  of  audit  units  has  been  divided  into  a  number  of  mutually  exclusive  strata,  a 
simple  random  sample  (without  replacement)  of  purchases  is  selected  from  each  stratum 
(1  .C).  These  samples  must  be  independent,  which  can  be  accomplished  by  use  of  different 
sets  of  random  numbers  for  the  selection  of  the  different  random  samples.  After  the 
samples  are  pulled  fi’om  the  strata,  they  are  analyzed  for  specific  attributes  or  values  (1  .D). 
Finally,  based  on  the  resuhs  of  the  analysis,  a  concluaon  is  drawn  about  the  original 
population (l.E)  (Neter,  1975,  p.  77). 


(A)  Heterogeneous 
population 


(B)  Heterogeneous 

population  divided  into 
homogenous  strata 


(C)  Samples  are  pulled 
from  strata 


(D)  Samples  analyzed  for 
specific  attributes 


(E)  Based  on  results  from 
(D),  projections  are 
made  about 
population  (A) _ 


.\  ^  ^ 
Cja^pl*  d^Ta> 


Figure  1.  Stratified  Sampling  Process 
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4.  Advantages  of  stratified  sampUng 

There  are  three  major  advantages  of  using  stratified  sampling: 

(1)  Efficiency.  Frequently  a  stratified  sample  can  produce  a  satisfiutory  result  with  a 
minimum  of  effort  and  eiqjense  when  conq}ared  to  a  simple  random  san^le.  One 
probability  sampling  procedure  is  more  ^dent  than  another  if  it  offers,  at  a  given  level  of 
confidence,  the  same  predsion  at  less  cost  or  greater  precision  at  the  same  cost.  For 
example,  a  hypothetical  sample  of  nine  teachers  could  be  observed  to  find  their  average 
years  of  education.  In  Table  I  their  variance  equals  2.61.  However,  when  teachers  are 
stratified  by  the  levd  of  school  t*  ''  teach  and  the  two  teachers  with  the  greatest 
difference  in  years  of  education  are  sampled,  the  variance  of  each  stratum  significantly 
declines.  These  variances,  which  measure  the  variability  of  years  of  education  within  each 
stratum,  are  much  smaller  than  the  variance  of  the  entire  population.  Consequently,  in  this 
instance,  a  stratified  random  sample  of  a  givoi  »ze  will  yield  a  substantially  more  precise 
estimate  than  a  simple  random  sample  of  greater  size,  since  much  smaller  variability  is 
encountered  within  each  stratum. 


Table  L  THE  EFFECTS  OF  STRATMCATION 


Teacher 

Level  of  School 

Years  of 
Education 

1 

Teacher 

Level  of  School 

Years  of 
Education 

1 

13.5 

6 

20.5 

2 

19.0 

7 

17.5 

3 

16.5 

1 

8 

wmsiiwm 

19.5 

4 

Elementary 

16.0 

■ 

9 

Elementary 

16.5 

5 

18.0 

Variance  =  2.61 

Stratum  1 ;  Elementary 

Stratum  2;  High  School 

Stratum  3;  College 

Teacher 

Years  of 
Education 

Teacher 

Years  of 
Education 

Teacher 

Years  of 
Education 

1 

15.5 

3 

16.5 

2 

19.0 

9 

16.5 

5 

18.0 

6 

20.5 

Variance  =  0.25 

Variance  =  0.56 

Variance  =  0.56 

Source:  Applied  Statistics,  Neter,  1993,  p.735 
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However,  for  stratified  sampling  to  be  efficient,  the  strata  must  be  designed  to 
contain  relatively  homogeneous  elements.  Homogeneity  is  accomplished  when  the  basis 
of  stratification  is  related  to  the  charactoistic  under  study  (e.g.,  level  of  school  taught  and 
years  of  education).  While  the  process  of  stratifying  a  sample  may  be  considered  an  extra 
effort  when  compared  to  taking  a  simple  random  sample  of  equal  total  size,  the 
stratification  process  normally  allows  for  a  decrease  in  the  total  sample  size  (see  Table  I). 
This  decrease  in  total  sample  size  is  desirable  because  precision  can  be  maintained,  or 
improved,  and  the  sampling  time  and  costs  are  reduced. 

(2)  Information  about  subpopulations.  Stratified  sampling  can  provide  secondary  findings 
of  strata  characteristics  in  addition  to  the  primary  findings  of  overall  population 
characteristics.  For  example,  a  study  by  a  university  of  the  effects  of  tuition  increases  may 
be  primarily  meant  to  provide  the  extent  of  sentiment  fi’om  all  students,  but  it  also 
provides  valuable  separate  information  about  graduate  and  undergraduate  students, 
minority  students,  male  and  female  students,  etc.  (Neter,  1993,  p.  735). 

(3)  Feasibility.  Sometimes  stratified  sampling  is  simply  the  most  feasible.  For  example,  if 
police  records  are  computerized  in  a  city  and  manual  in  the  rest  of  the  state,  administrative 
consideration  may  require  separate  sampling  in  the  city  and  elsewhere  in  the  state  (Neter, 
1993,  p.  735). 

B.  ESTIMATION  SAMPLING 

The  fimction  of  estimation  sampling  for  auditors  is  to  determine  two  things.  First, 
an  attempt  is  made  to  determine  the  number  of  occurrences  of  some  attribute,  such  as 
errors,  violations,  etc.  in  the  population.  Secondly,  once  the  attributes  are  identified,  it  is 
usually  appropriate  to  determine  the  m^nitude  of  those  attributes  in  order  to  make 
suggestions  for  improving  the  operation  under  review. 
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1.  Frequency  Estinuition  (Attributes) 

Often  it  is  sufiBcient  simply  to  determine  how  many  times  a  certain  attribute 
occurs.  Once  this  has  been  found  it  is  only  necessary  to  total  the  occurrences  beft>re 
making  conclusions  and  judgments  about  the  situation  that  was  observed.  This  is 
particularly  true  when  all  attributes  (errors,  violations,  etc.)  canty  equal  weight  and  one  is 
not  more  important  than  anotho'. 

During  the  DODIG  audit,  the  "attributes”  being  checked  were  the  occurrence 
of  excessive  (unnecessary  or  premature)  purchases.  Because  each  one  of  the  excessive 
purchases  observed  had  a  difierent  dollar  value,  they  were  not  sampled  equally.  Instead,  a 
higher  percentage  of  the  purchase  requests  in  the  Ugh  dollar  value  strata  were  selected. 

2.  Dollar  Value  Estimation  (Variables) 

Variables  estimation  is  the  statistical  method  used  to  estimate  dollar  values.  The 
objective  of  this  method  is  to  estimate  an  average  value  of  a  group  of  items  by  means  of  a 
sample  with  an  assurance  equivalent  to  the  confidence  level  that  the  sample  average  will 
be  within  a  range  of  some  specified  amount  fi’om  the  true  average  which  would  have  been 
attained  if  all  items  in  the  entire  population  had  been  averaged  (Hill,  1962,  p.  3 1). 

The  distinction  between  fi'equency  and  dollar  value  estimation  can  be  seen  when 
examining  invmitories.  On  the  one  hand,  a  company  may  want  to  check  the  reliability  of 
inventory  control.  Items  in  the  storeroom  are  identified  and  then  compared  to  the 
invemory  records  to  ensure  that  a  stock  record  exists  for  every  item.  On  the  other  hand, 
the  compare  may  want  to  determine  the  value  of  its  inventory.  A  sample  of  the  inventory 
would  be  counted,  priced,  and  extended.  On  the  basis  of  the  average  value  of  the  items 
selected,  the  aggregate  value  of  the  inventory  could  be  determined  within  the  limits  of 
reliability  attained  and  compared  to  the  compai^r’s  figures  (Hill,  1962,  p.  IS). 

In  the  case  of  the  DODIG  and  the  ICPs,  the  sample  purchase  requests  were 
analyzed  and  a  determination  was  made  as  to  whether  or  not  they  were  excessive. 
Secondly,  the  DODIG  determined  from  a  relatively  small  sample  of  purchase  requests  the 
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dollar  value  of  all  excessive  purchase  requests.  This  mettod  of  ddlar  value  estimation  will 
be  examined  more  closely  in  Chapters  in  and  IV. 

C.  CONFIDENCE  INTERVALS 

In  the  DODIG  report,  the  estimated  cost  savings  are  given  with  a  95  percent 
confidence  level  and  a  sampling  precision  of  ±  3%  (Jones,  February  1,  1993,  p.  33). 
Therefore  it  is  critical  to  know  exactly  what  this  means.  Confidence  intervals  are  used  for 
interval  estimation.  Interval  estimation  is  the  quoting  of  bounds  between  vdiich  the  true 
population  mean  lies.  This  is  appropriate  when  it  is  deared  to  0ve  some  indication  of  the 
accuracy  with  which  a  parameter  is  estimated  by  the  sample  (Rolnnson,  1992,  p.  120). 

In  any  situation  udiere  there  is  less  than  a  conq>lete  sample  (e.g.,  less  than  the 
entire  population),  there  ^  be  a  mar:^  of  uncertainty  surrounding  the  results.  This 
degree  of  uncertainty  may  be  measured  mathematically  and  expressed  in  terms  of 
reliability  (e.g.,  95%  confidence)  and  precision  (e.g.,  ±  3%).  Precision  and  reliability  are 
mathematically  interdependent  and  statistically  inseparable.  “The  precision  of  an  estimate 
indicates  the  range  within  which  it  is  expected  to  be  accurate,  and  the  reliability  (or 
confidence)  is  the  probability  of  achieving  this  accuracy”  (Newman,  1976,  p.  30). 

Probability  statements  caimot  be  made  about  a  single  event.  For  example,  if  a  coin 
is  flipped  once,  it  cannot  be  said  that  the  probability  of  the  coin  having  turned  up  heads  is 
one-half  The  coin  ather  turns  up  heads  or  it  does  not.  I^  however,  the  coin  is  flipped 
numerous  times,  a  meaningful  statement  can  be  made  that  the  relative  firequency  of  heads 
turning  up  approaches  0.5  (Cyert,  1962,  p.  8). 

The  reliability  statements  made  about  a  confidence  interval  are  similar  to  the 
statements  on  coin  flipping.  It  is  inaccurate  to  specify  the  probability  that  a  particular 
confidence  interval  contains  the  true  value  being  estimated.  However,  meaningful 
statements  can  be  made  about  the  probability  that  similar  confidence  intervals,  in  a  series 
of  sample  estimates,  contain  the  true  value  being  estimated.  For  e  iple,  it  might  be 
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appropriate  to  make  the  statement  that,  in  repeated  trials,  a  confidence  interval  of  |rfus  or 
minus  “x"  dollars  about  a  sample  estimate  will  contain  the  true  value  being  estimated  SK)% 
of  the  time  (Cyert,  1962,  p.  8). 

An  important  question  is  v«dun  aze  confidence  interval  should  be  used? 
answer  depends  on  the  tradeoff  between  the  confidence  levd  and  the  widt  he  interval. 

To  obtain  higher  confidence  fix)m  the  same  sample,  one  must  be  willing  to  du:ept  a  larger 
margin  of  error  (wider  interval).  The  way  to  attain  higher  confidence  and  still  have  a  short 
interval  is  to  take  a  larger  sample.  That  n»ans  that,  for  a  fixed  level  of  confidence  (e.g., 
95%),  the  confidence  interval  (e.g.,  ±  3%)  decreases  as  the  sample  size  increases.  Jr,  for 
a  give  interval,  the  confidence  level  inoeases  as  the  sample  size  increases  (Moore,  1979, 
p.  276). 

D.  STRATIFICATION 

The  final  bottom  line  impact  of  this  audit  conned  of  the  values  for  three 
measures  •  the  universe  of  contracts,  the  percoit  of  unnecessary  contracts,  and  the  Due  In 
Long  Supply  (OILS)  rate.  Each  of  these  is  fiuther  discussed  in  Chapter  m.F.  However, 
because  the  DILS  rate  is  determined  through  the  use  of  “stratification,”  this  term,  as  it 
applies  to  the  DILS  rate,  must  first  be  clarified. 

It  is  important  to  understand  that  we  are  talking  about  the  inventory  management 
definition  of  stratification  (also  called  “STRAT”)  which  is  an  entirely  separate  concept 
fi'om  the  stratified  sampling  process  de«nibed  earlier.  A  complete  description  of 
stratification  and  all  its  related  equations  is  beyond  the  scope  of  this  thesis.  However,  it 
should  be  understood  that  stratification  is  mainly  the  semi-aimual  process  of  comparing 
various  assets  and  requirements  levels  and  placing  each  in  a  priority  sequence.  The 


10 


stratification  program  is  used  to  analyze  each  inventory  hem.  After  all  items  have  been 
stratified,  the  total  stratification  budget  request  is  (ri>tained  by  adding  up  the  dollar  value 
of  future  expected  procuremoit/rqNur  defidendes  during  the  Budget  Year  (FMSO 
S230/S2A,  July  17,  1994,  p.  1-1).  Stratification  is  the  basis  for  budget  requests  needed  to 
operate  the  inventory  system. 

Because  stratification  tries  to  predict  future  needs,  h  uses  three  periods  over  which 
nuiterial  usage  must  be  considered;  Current  Year,  ^portionment  Year,  and  Budget  Year. 
To  initiate  the  forecast,  “STRAT*  first  looks  at  the  Opening  Portion  (the  “right  nov/’ 
inventory  and  requirements).  The  Openi'  r  Position  is  computed  by  taking  the  current 
system  assets  on  hand  and  matching  them  against  curroit  system  requirements  such  as 
safety  levels  and  lead  time  demand.  Next,  demands  and  forecasted  order  recdpts  are 
computed  for  the  Current  Year,  which  covers  the  remainder  of  the  fiscal  year  afiier  the 
Opening  Position  date.  The  ^portionment  Year  includes  the  twelve  months  following 
the  Current  Year  and  its  demands  and  forecasted  order  receipts  are  likewise  estimated. 
Finally,  the  Budget  Year’s  demands  and  required  orders  are  determined.  The  Budget 
Year  extends  fi-om  the  end  of  the  Apportionment  Year  through  to  the  following 
September  30th.  When  stratification  matches  assets  against  requirements,  it  may  not 
compute  material  requirements  but  rather  excess  inventory  and  may  therefore  recommend 
review  of  long  supply  conditions,  where  on  hand  and  due-in  material  exceeds  material 
retention  ceilings. 


11 


m.  PRESENTATION  OF  DATA 


A.  INTRODUCTION 

The  ot]jectives  of  the  DODIG  unfit  were  to  detemune  \^iether  quantities  of 
rq>airaUe  items  to  be  bou^  on  outstanding  {Mocurements  by  the  Navy  v^lesale  ICPs 
were  warranted  by  antidpated  requirements  and  wh^her  internal  management  controls 
over  the  determination  of  the  procurement  requirements  were  effiictive  (Jones,  February  1, 
1993,  p.l). 

>^^thin  the  last  five  years,  the  Dq>artment  of  Defense  Inqsector  Genoal  has 
changed  the  type  of  inventory  audits  conducted  in  the  services.  Previously  the  audits 
concentrated  primarily  on  Hems  that  were  newly  authorized  to  be  placed  in  inventory. 
Items  that  were  already  being  carried  in  stock  were  not  included  in  the  audit. 
Additionally,  procurement  lead  times  arrd  customer  demand  were  the  main  areas  reviewed 
(Interview  with  Mr.  J.  Chaney,  April  1994). 

In  1990,  DODIG  decided  to  take  a  slightly  dififerent  qrproach  to  their  audit.  At 
that  time  they  began  to  look  at  ail  hems  with  procurements  in  process,  whether  or  not  the 
procurements  were  for  new  hems.  Initially,  the  audits  were  to  be  DOD-wide.  However, 
DODIG  eventually  chose  to  audh  service  by  service  and  then  made  a  further  splh  between 
repairables  and  consumables. 

As  of  September  30,  1990,  the  ICPs  were  in  the  process  of  procuring 
approximately  $863  million  of  stock  for  1 1,308  depot  level  rqiairable  (DLR)  line  items 
(see  Column  (2)  in  Table  U  below).  The  process  to  purchase  these  items  starts  when 
automated  programs  are  triggered  to  reorder  and  the  program  recommends  a  purchase 
quantity  to  meet  the  stockage  objective.  The  inventory  manager  then  reviews  the 
requirement  and  other  relevant  data  to  verify  the  accuracy  of  the  computation.  Approved 
purchase  requests  serve  as  authorization  for  the  ICP’s  procurement  organization  to 
the  material. 
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Table  IL  SELECTION  OF  SAMPLE  UNIVB 

RSE 

(1) 

(2) 

(3) 

UNIVERSE 

PURCHASE  REQUESTS 
GREATER  THAN  $100,000 

ASO 

889 

SPCC 

541 

TOTAL  LINE  ITEMS 

HDim 

1430 

ASO 

$495,900,000 

SPCC 

$195,300,000 

TOTAL  DOLLAR  VALUE 

$691,200,000 

B.  AUDIT  SCOPE 

DODIG  initiaUy  intended  to  narrow  its  sample  universe  from  1 1,308  to  only  those 
procurements  valued  at  $100,000  or  greater  (Column  (3)  of  Table  II)  independent  of  ICP. 
This  cutoff  of  $100,000  was  chosen  by  DODIG  because  purchases  of  this  value  required 
greater  supervisory  review  and  checking  the  effectiveness  of  the  review  process  was  one 
purpose  of  the  audit.  At  the  same  time,  a  great  majority  of  the  total  doUar  value  of 
procurements  in  process  were  included  above  the  $100,000  level.  Specifically,  this 
decision  narrowed  the  population  to  1430  line  items  (13  percent  of  11,308)  but  stiU 
included  80  percent  of  the  total  value  of  the  procurements  in  process  ($691 .2  million). 


C.  AUDIT  SAMPLE 

From  the  universe  of  1430  line  itons,  initially  a  sample  of  107  line  items  was 
selected.  The  107  line  items  totaled  $229.4  million  in  procurements  (see  Column  (3)  of 
Table  III).  Appendix  B  ^ves  a  detailed  summary  of  all  purchase  requests  greater  than 
$100,000.  Before  the  sample  was  taken,  the  purchases  were  identified  by  ICP,  and  within 
ICPs  they  were  further  subdivided  by  size  of  procurement.  The  four  procurement  strata 
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were:  (1)  greater  than  $5  million,  (2)  $2.5  «  S  million,  (3)  SI  -  2.49  miilioii,  and  (4)  SlOO  - 
999  thousand. 


Table  m.  INITIAL  SAMPLE  SELECTION 


mnsm 

(2) 

(3) 

(4) 

PURCHASE 
REQUESTS  ^lEATER 
THAN  $100,000 

SAMPLE  FROM 
1430  ITEMS 

ITEMS  NOT 
QUALIFIED 
(DISCARI^) 

ASO 

889 

69 

15 

SPCC 

Ml 

38 

6 

TOTAL 

1430 

107 

21 

(7.5%  of  1430) 

ASO 

IHIHLLLlXildALLijHIHi 

SPCC 

TOTAL 

$691,200,000 

$41,100,000 

Of  the  107  herns  (listed  in  ^pendix  C),  21  did  not  "meet  the  criteria"  of  the 
DODIG  review.  These  21  items  (see  Column  (4)  of  Table  m)  were  excluded  from  further 
review  dther  because  the  purchases  wm  not  in  process  at  the  sanq}le  cutoff  date  (e.g., 
purchase  requests  were  canceled  or  contracts  had  already  been  awarded)  or  because  the 
purchases  were  for  items  that  were  managed  uring  "consumable  item"  management 
techniques.  It  was  necessary  for  audited  items  to  be  in  process  so  that  ai^  unnecessary 
procurements  identified  by  the  audit  could  be  cancded,  if  the  Navy  concurred  with  the 
findings.  In  reality,  SS5. 1  million  in  purchase  requests  were  canceled  by  the  ICPs  during 
the  audit.  Of  the  SSS.  1  million,  $33.0  million  were  initiated  by  the  ICPs  and  the  hardware 
systems  commands  (HSC),  independent  of  the  audit,  and  the  renuuning  $22. 1  million  were 
canceled  in  response  to  the  audit  (Jones,  Fd7ruary  1,  1993,  p.  2S).  Purchases  that  were 
excluded  because  they  were  treated  as  consumable  items  were  included  in  a  separate  audit 
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aidtled  “MUitaiy  Department  Requirements  for  Currently  Procured  Wholesale  Inventorfos 
for  Consumable  Items,”  DODIG  Report  No.  91-106  of  June  28,  1991. 

The  S41.1  million  item  adjustment  shown  in  Column  (3)  of  Table  IV  is  segmented 
by  ICP  and  stratum  in  .A4)pen(Ux  D.  This  adjustrnem  reduced  the  final  audit  samole  to  86 
line  hems  involving  purchases  valued  at  S188.3  million.  This  adjusted  sample  u  anown  in 
Column  (4)  of  Table  IV  and  is  shown  in  detail  in  ^pendbc  E. 


Table  IV.  REVISED  SAMPLE  SELECTION 


mmsm 

(2) 

(3) 

(4) 

SAMPLE  FROM 
1430  ITEMS 

ITEMS  NOT  QUALIFIED 
(DISCARDED) 

SAMPLED  ITEMS 
FOR  REVIEW 

ASO  69 

15 

54 

SPCC 

6 

21 

TOTAL 

107 

21 

86 

(80.4%  OF  107) 

ASO 

wmssmEssm 

$30,700,000 

$141,900,000 

SPCC 

TOTAL 

$41,100,000 

$188,300,000 

(82.1%of$229.4M) 

Based  on  the  previous  adjustments,  the  Navy  sample  universe  was  adjusted  to 
1056  line  items  with  purchases  valued  at  $520.2  million.  The  theory  behind  this  process  is 
discussed  in  Chapter  IV.E.  The  adjustment  of  the  sample  universe  was  done  in  two  parts  - 
by  line  items  and  by  dollar  value.  First,  the  number  of  line  items  of  the  new  sample  were 
compared  to  the  old  to  get  a  percentage  value.  This  value  was  then  multiplied  by  the  size 
of  the  old  universe  This  was  done  by  stratum  for  each  ICP  and  they  were  totaled  to  give 
the  number  of  line  items  for  the  adjusted  universe. 


IS 


Table  V  ASO  LINE  ITEM  ADJUSTMENT 


Stratum 

Old  Universe 

New  Universe 

1 

8 

7 

8 

7 

2 

20 

15 

26 

20 

4 

24 

17 

789 

559 

69 

54 

889 

644 

To  show  one  example.  Table  V  has  been  pulled  from  Appendix  B  and  ^pendix  E. 
The  calculation  of  the  adjustment  to  ASO  stratum  3  will  be  described.  First,  the  sample 
size  has  changed  from  seventeen  to  fifteen.  Fifteen  divided  by  seventeen  equals  .88235. 
When  .88235  is  multiplied  by  the  old  univo^  aze  (66),  the  new  universe  size  is  obtained 
for  ASO  stratum  3;  namely  58.  This  method  holds  true  for  each  ICP’s  stratum  and  finally 
gives  the  “adjusted  universe”  size  of  1056.  Secondly,  the  adjustment  to  obtain  the  “new 
universe”  value  is  calculated  by  getting  the  dollar  ^  of  the  itons  discarded  from  the 
sample  for  each  stratum,  dividing  it  by  the  origiiud  sample  value  and  then  multiplying  the 
quotient  by  the  old  universe  value.  As  an  example,  in  Table  VI,  stratum  3  is  calculated  by 
dividing  $2,601,156  by  $24,738,320.  The  quotient  (.1051)  is  then  multiplied  by 
$99,705,919  to  get  the  estimated  reduction  in  the  utuverse  value  of  $10,483,762.  This 
estimated  reduction  is  labeled  the  “projected  value”  in  Table  VI  as  that  is  the  term  used  by 
the  DODIG.  The  final  universe  value  is  determined  by  doing  this  for  each  ICP’s  stratum 
to  get  the  total  value  for  the  entire  adjustment  (items  deleted).  That  value  is 
$170,972,701.  When  this  adjustment  value  is  subtracted  from  the  original  universe  size, 
the  adjusted  universe  value  of  $520.2  million  is  obtained. 


Table  VI.  ASO  VALUE  ADJUSTMENT 


Stratum 

Adjustment  Value 

Old  Sample  Value 

Old  Universe  Value 

Prpjeaed  Value 

1 

$9,006,800 

$78,092,994 

■Eoomai 

■lESBEai 

2 

17,394,859 

64,160,301 

mmmkm 

22,737,337 

99.705»l 

s. 

SB 

4 

1.717.478 

5.648.744 

237.191.420 

72.117.091 

$30,720,293 

$498,856,170 

$1 14,344,990 
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D.  DODIG  AUDIT 

DODIG  auditors  examined  the  86  purchase  request  documents  related  to  the 
purchases  in  process  as  of  Septonber  30,  1990  to  evaluate  the  basis  for  procurement 
decisions.  They  evaluated  requirements  data  that  were  in  effect  at  the  time  of  the  audit  to 
determine  whether  requirements  supported  continuation  of  the  procuremoit. 

Specifically,  the  auditors  (1)  determined  whether  requirements  forecasts  were 
reasonable,  (2)  reviewed  the  accuracy  of  the  forecasted  denumd  rates,  (3)  evaluated  tl^ 
propriety  of  nondemand-based  (additive)  requirements,  and  (4)  verified  the  accuracy  of 
on-hand  asset  and  due-in  asset  balances.  Additionally,  the  auditors  selectivdy  reviewed 
other  requirements  data  and  other  fiictors  that  affected  the  requirements  forecast,  such  as 
past  and  future  program  data,  survival  and  wearout  rates,  and  rq>air  cycle  times  (Jones, 
Febriiary  1,  1993,  p.  2).  The  auditors  did  not  n^ely  consider  whether  the  purchases  were 
justified  at  the  time  of  reorder,  but  also  v«diether  the  initial  order  quantity  was  still  justified 
at  the  time  of  the  audit  in  September  1990.  Any  demand  changes  after  the  initiation  of  the 
audit  were  not  considered  part  of  the  audit  results.  Any  action,  or  inaction,  on  the  part  of 
inventory  managers  was  not  included  in  the  audit  results  (Interview  with  Mr.  J.  Chaney, 
May  1994). 

E.  DATA  CATEGORIZATION 

All  data  presented  in  this  section  ultimately  lead  to  the  DODIG’s  calculation  of 
projected  monetary  benefits  (fi'om  the  canceling  of  outstanding  orders)  derived  fi’om  this 
audit.  After  all  calculations  and  negotiations,  this  was  determined  to  be  S68.2  million. 
Reading  Table  VII  fi’om  left  to  right  shows  the  progression  to  reach  the  projected 
monetary  benefits  (listed  as  “PMB”).  First,  the  universe  is  divided  into  three  categories. 
Second,  the  excessive  purchase  requests  are  pulled  fi’om  the  universe  and  divided  into 
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three  categories.  Third,  unnecessary  purchase  requests  are  pull«i  fixrm  the  excessive 
purchase  requests  and  divicted  into  two  categories.  Finally,  the  (mrchase  requests  with  a 
“potential  monetary  benefit”  are  pulled  fi’om  tlM  unnecessary  purchases  to  detamirre  tlw 
findings  of  the  audit.  The  “other”  category  represents  ofl&etting  costs  that  reduce  net 
potential  benefits  (Jones,  February  1,  1993,  p.  47).  Each  one  of  the  eight  groupings, 
except  “Other,”  is  broken  down  by  ICP  and  stratum  in  Appendices  G  through  M. 


Table  m  SUMMARY  OF  AUDIT  RESULTS 


_ Q) _ 

(2) 

(3) 

Adjusted 

Universe 

Excessive 

Unnecessary 

Undetermin. 

$69,543,164 

$46,046,289 

$6$JS45w88T 

Reasonable 

Premature 

mmmm 

Other 

10.507.569 

The  first  step  after  the  doUar  value  of  the  adjusted  universe  was  determined  was  to 
divide  this  universe  dollar  value  imo  three  non-overiapping  categories  •  undeterminable, 
reasonable,  and  excessive.  The  total  combined  dollar  value  of  these  three  categories  must 
equal  the  dollar  value  of  the  entire  universe  ($523.2  million).  This  total  differs  fi'om  the 
previous  total  value  for  “adjusted  universe”  ($520.2  million)  described  in  Section  C  and 
the  reason  for  the  difference  will  be  discussed  in  Chapter  IV.E. 

1.  Undeterminable 

The  first  category  into  which  the  universe  was  subdivided  was  called  “unde¬ 
terminable.”  Undeterminable  included  all  purchase  requests  for  which  DODIG  was  unable 
to  determine  the  reasonableness  of  purchases  because  the  ICPs  could  not  provide 
verifiable  requirements  data  as  of  September  30,  1990  or  the  requirements  were  dependent 
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on  a  management  decision  by  an  HSC  (Jones,  February  1,  1993,  p.  34).  Purchase  requests 
were  considered  “eccessive”  if  the  quantity  ordered  exceeded  the  DODIG-defined 
“stockage  objective”  by  more  than  twdve  months  of  forecast  requiranoits.  Purchase 
requests  that  did  not  fit  into  either  of  the  these  categories  were  consid«^  “reasonable.” 

The  method  to  make  the  projection  based  on  the  sample  is  idoitical  for  each  of 
these  three  categories.  First,  the  quantity  (hits)  and  dollar  values  (hit  values)  are 
determined.  A  purchase  request  is  considered  a  “hit”  if  it  either  wholly  or  partially  fits 
into  one  of  the  three  categories.  For  instance,  one  purchase  request  may  be  partially 
reasonable  and  partially  excess.  However,  the  dollar  value  of  the  two  groupings  cannot 
exceed  the  original  purchase  request  value.  For  instance,  a  purchase  request  for  $1 10,000 
may  be  determined  to  be  $10,000  excess  by  the  auditors.  This  one  line  item  would  then  fit 
the  “reasonable”  category  with  a  “hit”  value  of  $100,000  and  the  “excessive”  category 
with  a  “hit”  value  of  $10,000.  Although  the  term  “hit”  implies  a  problem,  the  DODIG 
uses  the  term  simply  to  classify  purchase  requests  and  their  dollar  values. 


Table  VIIL  ASO 

UNDETERMINABLE  LINE  ITEM  PROJECTION 

Stratum 

Hits 

Universe  Quantity 

il£S55gil.{ia 

1 

2 

8 

8 

2 

2 

4 

20 

26 

5 

3 

66 

12 

4 

1 

24 

789 

M 

11 

69 

889 

85 

To  make  the  projection,  the  hit  quantity  is  divided  by  the  sample  quantity  and 
that  quotient  is  then  multiplied  by  the  universe  quantity.  The  answer  is  then  rounded  to 
the  nearest  integer.  Excerpts  from  Appendices  B  and  G  are  given  as  examples.  In  Table 
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VIII,  ASO  stratum  3  shows  three  undeterminable  items.  Three  is  then  divided  by  the 
sample  quantity  of  seventeen.  The  quotient  (.17647)  is  then  multiplied  by  the  universe 
quantity  of  66  to  get  twelve  projected  hits  for  stratum  3.  The  same  methodology  is  used 
to  get  the  projected  values  in  Table  DC.  Specifically,  $4,646,285  divided  by  $24,738,320 
equals  .187817321.  Then  the  universe  value  of  $99,705,919  multiplied  by  .187817321 
equals  $18,726,500  (the  projected  value  for  ASO  stratum  3). 


Table  DC.  ASO  UNDETER] 

MINABLE  VALU 

E  PROJECTION 

mm 

(2) 

_ m _ 

(4) 

(5) 

Stratum 

Hit  Value 

Universe  Value 

1 

2 

mmmm 

4 

389.754 

5.648.744 

237.191.420 

16.365.837 

mMmmm 

$172,640,358 

$498,856,170 

$69,543,164 

A  summary  of  the  projected  hits  and  projected  values  for  unr' terminable, 
reasonable,  and  excessive  purchase  requests  is  shown  in  ^pendices  G  through  I.  The 
total  projected  values  of  these  three  categories  does  equal  $523,168,692  as  shown  in 
Table  VH. 

2.  Excessive 

The  next  major  step  the  DODIG  took  to  determine  projected  monetary  benefits 
was  to  break  down  the  “excessive”  category  into  three  subcategories.  The  DODIG  calls 
these  groups  non-projectable,  premature,  and  urmecessary.  The  total  of  these  three  areas 
should  equal  $156,563,299,  wluch  is  the  total  for  the  “excessive”  category  as  shown  in 
Appendix  I  and  Column  (1)  of  Table  VII.  However,  a  thorough  review  of  all  working 
papers  shows  that  these  three  actually  total  $156,923,374  -  a  difference  of  $360,075 
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(Barton,  Nov.  16, 1992,  summary  page).  The  impact  of  this  difference  will  be  discussed  in 
Chapter  IV.E. 

The  method  used  to  project  the  number  of  hits  and  the  hit  value  in  the  univo^ 
was  the  same  as  that  used  to  make  projections  for  the  adjusted  universe  described  in  the 
previous  section.  The  number  of  hits  in  the  sample  stratum  are  divided  by  the  stratum 
sample  size  and  the  quotient  is  multiplied  by  the  universe  size.  The  product  is  then 
rounded  to  the  nearest  whole  number.  This  is  then  done  for  each  stratum  and  the  results 
are  totaled  for  each  ICP  to  get  the  projected  hits  and  projected  total  hit  value. 

3.  Unnecessary 

The  final  category  that  the  DODIG  calculated  was  the  “projected  monetary 
benefit”  portion  of  the  purchase  requests  determined  to  be  unnecessary.  The  DODIG 
projected  $68.2  million  of  potential  monetary  benefits  fi’om  the  unnecessary  items  (see 
Table  VII).  The  $10.6  million  difference  ($78.8  million  total  uimecessary  minus  $68.2 
million)  represents  offeetting  costs  to  repur  unserviceable  assets  (Jones,  Fd)ruary  1,  1993, 
p.  47).  However,  the  audit  report  does  not  explain  the  specific  nature  of  those  costs  or 
how  or  why  they  ofiset  benefits  fi’om  avoiding  unnecessary  purchases.  The  proje  on  of 
the  potential  monetary  benefits  can  be  seen  in  Appendbc  M.  The  methodology  used  to 
make  the  projection  is  also  identical  to  the  calculations  used  to  nuike  projections  for  the 
adjusted  universe  as  described  above  in  section  E.  1. 

F.  AUDITIMPACT 

The  actual  firud  impact  of  this  audit  was  reported  in  an  NC-2  Report  submitted  by 
the  Naval  Supply  Systems  Command  (NAVSUP)  with  the  reapportionment  budget  to  the 
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Navy  Comptrollo'  (NAVCOMPT).  This  report  summarized  all  audit  findings  and 
recommendations  fi'om  the  previous  fiscal  year.  The  purpose  of  this  report  was  to  ensure 
that  monetary  benefit  claims  (iHidget  reduction)  are  actually  incmporated  into  the 
following  year’s  budget. 

The  bottom  line  for  this  particular  DODIG  audit  was  calculated  using  three 
measures: 

(1)  12  (universe)  -  $520. 1  million.  The  value  of  the  adjusted  universe  size  reported  in  the 

audit. 

(2)  Q  (unnecessary  contracts)  -  71%.  The  audit  projected  SI  10.9  million  in  premature  or 

unnecessary  procurements  (see  Table  VII).  Unnecessary  contracts  were  projected 
to  total  $78.8  million  (or  71%  of  $1 10.9  million). 

(3)  D  (DBLS)  -  8.23%.  The  Due  In  Long  Supply  (DILS)  rate  calculated  during  each 

semiannual  STRAT  (as  described  in  Chapter  n.D).  NAVSUP  figured  this  DILS 
rate  by  estimating  the  total  value  of  all  contracts  in  excess  of  known  requirements 
during  the  Budget  Year  and  dividing  this  figure  by  the  total  value  of  contracts 
expected  to  be  on  order  during  the  Budget  Year.  The  average  DILS  rate  firom 
September  1990  through  September  1991  was  8.23%. 

To  obtain  the  figure  that  NAVSUP  used  to  report  to  NAVCOMPT,  the  following 
equation  was  used:  U  x  C  x  D  =  Agreed  Potential  Monetary  Benefit.  NAVSUP  reported 
this  agreed  potential  monetary  benefit  to  be  $30.4  million  or  $520.  IM  x  71%  x  8.23%. 
This  figure  was  used  by  NAVSUP  in  their  arbitration  hearing  with  the  DODIG.  The 
arbitrator  was  a  representative  from  the  Office  of  the  Undersecretary  of  Defense  for 
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Logistics  (OUSD(L)).  OUSD(L)  determined  that  $30.4  million  was  accq>table  for 
NAVSUP  to  report  to  NAVCOMPT.  Furthermore,  because  the  $30.4  million  was 
classified  as  a  one  time  “cost  avoidance”  for  fiscal  year  1991  rather  than  “savings”  fi'om 
the  audit,  NAVSUP  was  not  required  to  incorporate  this  figure  into  their  budget  estimate 
for  fiscal  year  1994  but  rather  simply  to  list  it  as  a  recognized  audit  finding.  In  other 
words,  a  cost  would  have  occurred  if  the  audit  had  not  taken  place.  However,  because 
there  was  an  audit,  the  costs  are  said  to  have  never  occurred. 
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IV.  DATA  ANALYSIS  AND  INTERPRETATION 


A.  INTRODUCTION 

This  chapter  indudes  a  critique  of  the  entire  pnocess  used  by  the  DODIG  to  make 
conclusions  found  in  the  audit  report.  Specifically,  Section  B  discusses  the  DODIG 
selection  of  the  universe  aze  and  sample  size,  strata,  confidence  levd,  and  precision.  This 
infoimation  was  obtained  fix)m  their  audit  report,  the  audit  working  papers  provided  by 
the  Quantitative  Methods  Division  of  the  DODIG,  and  discussion  with  the  statisticians 
assigned  to  the  audit.  Section  C  begins  tlw  analysis  of  the  DODIG  dedaons  and  then 
presents  an  alternative  method  of  sdecting  the  sample  size  along  with  two  ways  of 
dividing  the  sdected  sample  over  the  chosen  strata.  Section  D  provides  a  method  to  tea 
stated  levels  of  confidence  and  precision.  The  chapter  concludes  with  a  discussion  of 
several  inconaaencies  in  the  reported  figures  used  during  the  audit. 

B.  DODIG  STATISTICAL  DECISIONS 

1.  Adjusted  Universe 

As  mentioned  in  Chapter  in.C,  the  DODIG  treaed  hems  tha  they  did  not  want 
to  include  in  the  audit  in  the  same  way  as  the  attributes  they  did  want  to  indude.  For 
example,  when  their  samples  uncovered  excessive  or  reasonable  purchase  requ^s, 
projections  were  used  to  calculae  the  number  and  vdue  of  these  purchase  requeas  in  the 
universe.  At  the  same  time,  when  the  DODIG  found  sampled  purchase  requeas  tha  they 
daermined  were  outside  the  scope  of  the  audit,  they  used  the  same  projection  procedures 
to  “adjua”  the  universe.  While  the  process  would  have  been  time  consuming,  the  only 
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statistically  valid  way  to  ‘‘adjust”  the  universe  is  by  going  through  the  original  universe  and 
discarding  the  purchase  requests  deemed  to  be  outside  the  scope  of  the  audit.  Only  after 
this  is  complete  can  a  true  sample  be  used  to  make  predictions  about  the  population  of 
relevant  purchase  requests. 

Specifically,  the  initial  sample  of  107  items  found  21  items  that  were  determined 
to  be  outside  the  scope  of  the  audit.  Based  on  these  21  items,  the  DODIG  determined  it 
li  :ely  that  there  was  a  amilar  proportion  of  items  in  the  entire  population  of  1430.  The 
projection  calculated  374  total  population  itons  outade  the  audit  and  therefore  the 
population  was  “adjusted”  to  1056  items.  A  similar  projection  found  SI  71  million  of  the 
original  population  value  to  be  outside  the  audit’s  scope.  Therefore,  the  population  value 
was  reduced  from  $691 . 1  million  to  $520. 1  million.  Both  of  these  projections  were  made 
the  same  way  the  projections  were  made  for  every  other  grouping  used  during  the  audit. 
This  procedure  is  described  in  Chapter  in.E.  The  results  of  this  projection  are  then  shown 
in  Appendix  D.  Again,  the  only  trxily  valid  way  to  adjust  the  population  is  to  go  item-by¬ 
item  through  the  population  and  eliminate  items  outside  the  audit’s  scope.  Only  when  this 
process  is  complete  can  a  usable  sample  be  sdected. 

However,  the  DODIG  did  not  actually  use  the  “new”  universe  of  1056  items 
valued  at  $520.1  million.  Instead  the  original  universe  size  and  value  were  used 
throughout  the  entire  audit  and  in  all  calculations.  The  DODIG’s  actual  purpose  of 
determining  the  “adjustment”  was  to  ensure  the  total  value  of  those  items  which  were 
outside  the  audit’s  scope  was  set  aside  and  not  mingled  with  any  other  part  of  the  audit. 
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Therefore,  the  audit  results  did  not  become  flawed  as  they  may  have  foe  "a^luated 
universe  vahie”  was  actually  used  in  calculatkms. 

2.  Sample  Siw 

As  previously  discussed,  the  chosen  samfrfe  size  for  tins  audit  was  107  out  of 
1430  Hems.  Deqnte  the  foct  that  this  sudH  of  the  Navy  conunands  was  similar  to  nidits 
the  DODIG  conducted  with  the  Air  Force  and  Army,  foere  were  no  sanqde  size  sdectkm 
procedures  available  for  review.  Once  the  dedsimi  to  sample  107  hems  was  made,  foere 
was  no  algorithm  used  to  divide  the  107  into  the  four  strata  (Interview  with  Dr.  D. 
Barton,  May  1994).  Instead,  consdous  dedsions  were  made  to  make  100%  reviews  of 
the  top  dollar  value  stratum,  sample  iq;>|noximatdy  80%  of  the  second  stratum,  and  then 
divide  the  remaining  hems  over  the  bottom  two  strata.  Once  the  decision  was  made  to 
sanqjle  percent  from  each  stratum,  the  purchase  requests  were  chosen  by  using  simple 
random  sanq)ling. 

3.  Stratification 

One  of  the  audhms’  first  decisions  after  dedding  to  sample  only  those  purchase 
requests  valued  at  $100,000  or  larger  was  to  stratify  the  universe  of  purchase  requests  into 
four  strata.  As  mentioned  in  Section  B.2,  the  stated  purpose  of  the  strata  sizing  decisions 
was  to  ensure  a  review  of  a  high  percentage  of  the  highest  dollar  value  purchase  requests. 
In  other  words,  the  top  strata  should  not  include  too  many  hems.  The  strata  breaks  were 
based  partially  on  “natural  breaks”  in  the  universe  of  purchase  requests  and  partially  on  the 
level  of  supervisory  review  required  for  the  purchase  requests  (Intoview  with  Mr.  J. 
Chaney,  May  1994). 


One  method  to  verify  the  effectiveness  of  stratification  is  to  ctHnpare  the 
standard  deviation  of  the  san^les  before  and  after  stratification.  Secondly,  the  coefficient 
of  variation  diould  be  detennined  to  see  the  rdative  variability.  Equation  (1)  in  Appendix 
A  was  used  to  make  that  calculation.  As  can  be  seen  in  Table  X,  simply  dividing  the 
sample  between  ASO  and  SPCC  does  not  decrease  the  rdative  variability  -  SPCC’s 
coefficient  of  variation  (136.74)  is  higher  than  the  combined  coefficient  of  variation 
(128.71).  The  true  advantage  of  stratification  is  apparait  whoi  the  sample  is  broken  imo 
strata  by  the  size  of  the  purchase  request.  This  can  be  seen  by  the  decrease  in  the 
coefficients  of  variation  once  the  stratifications  are  made. 


Table  X.  SUMMARY  OF  STANDARD  DEVIATIONS  OF  PURCHASE  REQUEST 
VALUES  AND  COEFnCIENTS  OF  VARIATION 


Standard  Deviation  (S) 

Coefficient  of  Variation  (Pet) 

ASO  and  SPCC 

2,759,542.57 

128.71 

ASO 

3.038,801.45 

121.45 

ASO  -  Stratum  1 

2,494,853.84 

25.56 

ASO  -  Stratum  2 

655,963.22 

20.45 

ASO  -  Stratum  3 

378,072.98 

25.98 

ASO  -  Stratum  4 

128,973.78 

54.80 

SPCC 

2,042,441.49 

136.74 

SPCC  -  Stratum  1 

2,525,926.84 

30.07 

SPCC  -  Stratum  2 

420,372.86 

13.02 

SPCC  -  ’atum3 

503,727.75 

32.51 

SPCC  -  um  4 

172,256.03 

67.72 

4.  Confidence  Level  and  Precision 

An  important  statement  in  the  report  of  audit  findings  was  that  “the  sample 
results  were  projected  with  a  95%  confidence  level  and  a  sampling  precision  of  ±  3 
percent  for  dollars”  (Jones,  February  1,  1993,  p.  33).  In  terms  of  the  audit  results,  this 


translates  to  a  $68,245,887  finding  with  a  95  percent  confidence  and  a  precision  of  less 
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than  ±  S2,047,377.  In  the  audit  wt^king  papers  the  **niafBin  oi  error”  is  ttated  as 
$552,396.  Clearly,  this  is  within  the  stated  precision.  However,  dieee  figures  are  analyzed 
in  Section  D  to  test  their  accuracy. 

C.  SAMPLE  SIZE  ANALYSIS 
1.  Precision 

Before  attempting  to  verify  the  predsitm  of  ibt  audit  results,  equations  (2) 
through  (4)  in  ^pendix  A  can  be  used  to  obtain  a  confidence  interval  for  the  projected 
number  of  hits  made  by  the  DODIG  (sera  in  Appradbc  M).  First,  it  is  necessary  to  obtain 
the  population  proportion.  Secondly,  an  estimated  variance  riiould  be  determined.  Then 
the  variance  can  be  used  to  obtain  a  confidence  interval  at  the  95%  confidence  level.  The 
step-by-step  process  is  shown  bdow. 

a.  Estimator  of  the  Population  Proportion 

In  order  to  estimate  the  proportion  (fiaction)  of  purchase  requests  that  were 
unnecessary,  with  projected  monetary  benefits,  the  DODIG  auditors  used  equation  (2)  in 
Appendix  A.  The  population  was  divided  into  strata  and  a  sample  was  taken  fi’om  each 
stratum.  The  purchase  requests  were  then  thoroughly  analyzed  to  see  which  were 
considered  uimecessary  with  projected  monetary  benefits.  The  following  calculations 
show  how  DODIG  obtained  the  projected  quantities  for  ASO  and  SPCC,  respectively. 
These  figures  match  those  sera  in  ^pendix  M. 

First,  the  formula  for  equation  (2)  is; 

”  l>l 


Z8 


where  N  is  the  total  universe  size,  Ni  is  the  universe  size  by  stratum  as  shown  in  Appendix 
B,  and  is  the  proportion  of  the  sample  that  was  found  to  be  unnecessary  with  a 
projected  monetary  benefit.  Next,  values  of  these  parameters  were  substituted  imo  the 
equation. 

ASO:  p.  -  j^[8(0)+26(.15)+«6(.1765)+78<»(.25)] 

-.2394 

SPCC:  p.-^[2(0)+6(.1667)+21(.3)+512(.35)] 

-^[186.5502] 

=  .3447 


For  example,  15  (above)  for  ASO  and  was  the  ratio  3/20  where  3  was  the  number  of 
hits  for  ASO  stratum  2  and  20  was  the  sample  size  for  ASO  stratum  2.  The  p^  values 
were  computed  fi'om  the  number  of  hits  in  ^pendix  M  and  the  stratum  sample  sizes  in 
Appendix  B. 

The  .2394  and  .3447  figures  are  used  in  Subsection  c  to  determine  the 
bounds  on  the  errors  of  estimation.  Also,  the  summations  of  N^p^  (212.797  and  186.55 
for  ASO  and  SPCC,  respectively)  show  how  the  DODIG  projected  the  number  of 
unnecessary  purchase  requests,  listed  in  Appendix  M. 
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b.  Estinatcd  Variaace 


The  estimated  variaiKe  can  be  (4>taiiied  by  using  equatkm  (3)  from  Appendix 
A.  The  estimated  variance  is  needed  to  olMain  the  confidence  interval  shown  in 
Subsection  c  bdow. 


Calculating  this  solution  is  simplified  by  sq)arateiy  calculating  f  ^  |  1,  which  is 

V  Nt 

also  equal  to  nA)-  VtPi)is  determined  as  follows; 


^66-17^ 

■(.1765X.8235)] 

1  66  4 

i«  J 

("789-24 

j(.25X.75)] 

1  789 

A  23  J  • 

=  006744356; 


Substituting  these  values  into  equation  (3)  of  ^pendix  A  gives: 

V(p^  )=  + (26)'  (. 001 55) + (66)'(. 00674) + (789)'(. 0079)] 


4950.955 

(889)' 


■006264486. 


Similarly  for  SPCC: 


V(p,)=0; 

V(A)-0; 
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and,  therefore 

^Pm)  -  ^^[0+0+(2l)*(.012)+(512)*(.01 1506)] 


3021.608 

(541)* 


=  0103239. 


c.  Bound  on  Error  of  Estinuition 

The  results  obtained  from  Subsections  a  and  b  are  then  used  with  equation 
(4)  from  .44>pendix  A  to  obtain  a  95%  confidence  levd.  For  ASO,  the  95%  confidence 
interval  for  the  finction  of  purchase  requests  having  projected  monetary  benefits  is  given 
by 

±2^y{pJ  =  .2394  ±  2 V. 006264486 
.23941.158297016 


or  .0811^^^^.3977 


Similarly,  for  SPCC  the  95%  confidence  interval  is 

.344712V.010323899 

.34471.203213174 


or  .1415^^^^.5479 

From  Section  2  and  Appendix  M,  the  estimated  number  of  unnecessary 
purchase  requests  for  ASO  and  SPCC  were  213  and  186,  respectively.  Using  the  results 
from  this  section,  the  95%  confidence  interval  for  ASO  would  be  from  72  (889  x  .081 1)  to 
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354  (889  X  .3977).  For  SPCC,  the  confidaKe  interval  is  fitmi  77  to  296.  The  sixe  of 
these  intervals  is  quite  large.  “We  could  rechice  this  bound  and  nuke  the  estinutor  more 
precise  by  imareasing  the  sample  size”  (Mendenhall,  1990,  p.  1 18). 

In  an  effort  to  get  a  tighter  boumi  on  the  errors  of  estimation,  the  strata  for 
ASO  and  SPCC  were  combined  and  the  calculations  wen  again  made  to  determine  a  9554 
confidence  interval.  The  aggregate  variance  decreased  to  .00388  fit>m  .00626  (for  ASO) 
and  .01032  (for  SPCC).  However,  the  range  was  still  fiom  230  (1430  x  .161)  to  586 
(1430  X  .41).  The  range  is  less  than  it  is  for  ASO  and  SPCC  calculated  separately. 
However,  the  bounds  on  the  number  of  estimated  unnecessary  purchase  requests  would 
still  be  considered  large. 

2.  Selecting  Sample  Size  and  Stratification 

The  previous  section  looked  at  how  much  precision  existed  for  the  projected 
number  of  hits  found  by  the  DODIG.  This  section  considers  methods  to  increase  the 
precision  of  sample  hits  along  with  guidelines  to  subdivide  the  chosen  sample  size  among 
the  given  strata.  The  two  options  are  based  on  allocating  the  sample  size  among  the  strata 
:n  proportion  to  the  universe  size  and  then  in  proportion  to  the  universe  value, 
a.  Sample  Size  Selection 

In  order  to  ensure  increased  reliability  fi’om  sampling,  methods  have  been 
used  to  select  sample  sizes  based  on  (1)  the  expected  rate  of  occurrence  of  the  specified 
characteristic,  (2)  the  desired  confidence  level,  (3)  the  number  of  items  in  the  universe,  and 
(4)  the  desired  reliability,  .^pendix  N  was  taken  fi’om  (Hill,  1962)  and  has  taken  into 
consideration  each  of  these  foctors.  Its  values  are  based  on  numerous  calculations  of 
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equation  (S)  in  Appendix  A.  When  the  rate  of  occurrence  is  expected  to  be  above  10%, 
the  table  with  a  50%  rate  of  occurroice  is  used  to  select  a  sample  size.  In  this  audit,  the 
projected  rate  of  occurrence,  as  given  in  .^pendix  M,  turned  out  to  be  28%  (399  1430). 

Finally,  the  sample  size  would  have  been  624  if  the  auditors  were  only  looking  at  sample 
attributes  and  not  sample  values. 

b.  Proportional  Sample  Sizes 

This  section  and  section  c  assume  that  the  sample  size  of  107  is  correct  and 
proposes  two  processes  to  spread  the  107  over  the  four  strata.  In  actuality,  the  sample 
size  used  by  the  DODIG  was  not  proven  correct.  But,  because  the  procedure  to  choose 
the  107  items  was  not  available  for  review,  this  sample  size  will  be  the  one  allocated  over 
the  four  strata. 


Table  XL  THE  EFFECT  OF  PR( 

DPORTIO 

NALSAMl 

PLE  SIZES 

(1) 

(2) 

(3) 

(4) 

_ m _ 

(6) 

Stratum 

Universe 

Size 

Percent  of 
Universe 

Sample 

Size 

ASO-  1 

8 

0.9 

8 

11.6 

1 

2 

26 

2.9 

20 

29.0 

2 

3 

66 

7.4 

17 

24.6 

5 

4 

789 

24 

34.8 

61 

889 

100.0 

69 

100.0 

69 

SPCC-  1 

2 

0.4 

2 

5.3 

1 

2 

6 

1.1 

6 

15.8 

1 

3 

21 

3.9 

10 

26.3 

1 

4 

512 

20 

32.6 

35 

341 

100.0 

38 

100.0 

38 

One  possibility  is  to  spread  the  sample  over  the  strata  in  proportion  to  the 
universe  size  (Cochran,  1963,  p.  89).  Table  XI  shows  what  percentage  of  the  universe 
was  in  each  stratum  and  then  what  percentage  of  the  sample  was  in  each  stratum.  Column 
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(3)  shows  that  for  both  ASO  and  SPCC  i^proximately  90%  of  the  universe  items  are  in 
the  low  dollar  value  category.  Column  (4)  shows  the  sample  sizes  initially  selected  for 
each  stratum.  Colunm  (S)  shows  the  percentage  that  each  stratum  sample  was  of  the 
whole  for  each  ICP.  If  the  sample  size  had  stayed  at  107  and  was  split  between  ASO  and 
SPCC  as  in  the  audit  and  the  sample  had  been  spread  over  the  strata  in  the  same 
percentages  as  the  four  strata  were  of  the  universe,  then  the  san4>le  sizes  would  have  been 
the  spread  seen  in  Colunm  (6).  However,  this  sample  stratification  would  not  have  met 
the  stated  purpose  of  capturing  a  high  percentage  of  the  universe’s  value  with  the  smallest 
possible  sample. 

c.  Proportional  Sample  Values 

To  reflect  the  goal  of  sampling  a  high  percentage  of  the  dollar  value  of  the 
universe,  the  sample  per  stratum  could  have  been  determined  by  using  the  percentage  of 
the  universe  value  contained  within  each  stratum  (see  Column  (3)  of  Table  XII).  This 
method  would  avoid  results  like  those  currently  seen  in  the  fourth  strata,  where 
approximately  S0%  of  the  universe  value  is  included  but  less  than  10%  of  the  sample 
value.  When  Column  (3)  is  used  to  determine  the  appropriate  sample  size  within  each 
strata,  the  adjusted  sample  size  (Colunm  (6))  is  not  nearly  as  skewed  to  include  the  high 
dollar  value  strata  as  the  actual  sample  itself  (Colunm  (7)).  However,  the  top  dollar  value 
stratum  sample  sizes  do  not  change  for  either  ASO  or  SPCC.  Instead,  the  two  middle 
strata  have  smaller  samples  and  the  lowest  dollar  value  strata  sample  sizes  increase. 
Column  (6)  is  calculated  by  multiplying  the  column  (3)  percentage  by  the  original  ICP 
total  sample  size.  For  example,  ASO  stratum  3  is  calculated  by  multiplying  20.1%  by  69 
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to  obtain  14.  When  the  product  exceeds  the  entire  stratum  population  as  in  SPCC  stratum 
1  (8.6%  X  38  =  3),  then  the  difference  (1)  is  added  to  the  next  highest  stratum.  This 
decision  ensures  the  emphasis  remains  on  the  highest  dollar  value  strata,  as  in  the  DODIG 
sampling  plan.  While  this  sample  distribution  is  closer  to  the  actual  sanq)le  than  that  seen 
in  Section  b,  it  still  emphasizes  the  fourth  stratum  more  than  the  actual  cUd. 


Table  XIL  THE  EFFECT  OF  PROPORTIONAL  SAMPLE  VALUES 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

■KJBH 

Stratum 

Univeise 

Percent  of 
Universe 

Sample 

Percent  of 
Sample 

Adjusted 

Sample 

Actual 

Sample 

ASO-1 

$78,092,994 

15.8 

$78,092,994 

45.2 

8 

8 

2 

83.895.837 

16.9 

64.160.301 

37.2 

14 

20 

4 

234.191.420 

5.648.744 

33 

24 

100.0 

172.640.359 

100.0 

69 

69 

SPCC-1 

8.6 

16.801.000 

29.6 

2 

2 

2 

19.377.237 

9.9 

19.377.237 

34.1 

5 

6 

3 

34.665,847 

17.8 

15,493.707 

27.3 

7 

{HEHMH 

4 

124.441.089 

Ql 

5.087.281 

24 

20 

195.285.173 

100.0 

100.0 

38 

38 

D.  CONFIDENCE  LEVEL  AND  PRECISION  ANALYSIS 

When  statements  are  made  about  a  sample  result’s  confidence  level  and  predsion, 
it  is  important  to  be  able  to  test  these  measures.  Using  a  procedure  used  by  Des  Raj  in 
The  Design  of  Sample  Surveys,  this  section  does  exactly  that.  Equations  (1),  and  (6) 
through  (9),  in  Appendix  A  are  used  here  to  determine  the  precision  of  the  stated  results, 
given  the  stated  95%  confidence  level.  Both  SPCC  and  ASO  are  combined  to  ensure  that 
the  tightest  possible  interval  is  obtained. 

The  first  step  is  to  gather  the  data  shown  in  Table  Xm.  Columns  (2)  and  (3)  of 
Table  XIII  are  taken  from  Appendix  B.  Column  (4)  can  be  calculated  by  dividing  the  total 
stratum  hit  value  from  Appendix  M  by  the  sample  size  (CoIumn(3)).  For  example,  stratum 
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2  (S2I0,006)  is  calculated  by  dividii^  $5,460,151  by  26.  Cohnmi  (5)  is  calculated  iraiQ 
equation  (6)  from  Appendix  A.  When  TaUe  Xm  is  cmnplete,  the  populatkm’s  UMal  hit 
value  is  estimated.  Using  equation  (7)  from  ^)pendbc  A; 

L 

=  10(0)  +  32(210,006)  +  87(209,250)  +  1301(32,441) 

is| 

=  $67,130,830 


Table  Xn 

IL  DATA  TO  DETERMINE  PRECISION 

WBsm 

'■am 

■m 

(4) 

(5) 

Stratum 

Universe 

(K) 

Sample 

("l) 

HhMean 

Hit  Variance 
(^) 

1 

10 

10 

0 

2 

32 

26 

3 

87 

27 

4 

44 

WEmmm 

1430 

107 

This  figure  is  less  than  the  DODIG  figure  of  $68,245,847  shown  in  Appendix  M. 
Secondly,  equation  (8)  gives  the  mean  value  per  purchase  request  of  $46,945.  Third, 
equation  (9)  gives  the  following: 

=  2,603,318,149,504  +  40,077,250,485,280  +  237,516,709,061,100 
=  $280,197,277,695,884 

When  this  result  is  divided  by  the  square  of  the  population  (1430),  the  variance  of  the 
estimate  is  determined  to  be  $137,022,484  and  the  standard  deviation  $1 1,706. 

Using  the  procedures  outlined  in  The  Design  of  Sample  Surveys,  the  next  step  is  to 
determine  the  coefficient  of  variation  using  equation  (1)  in  Appendix  A; 
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=  24.935033% 

The  coefiBdent  of  variation  multiplied  by  the  estimated  total  hit  value  of  $67,130,830 
equals  $16,739,094.  This  figure  is  then  used  in  equation  (10)  of  Appendix  A: 

_  16,739,094 
=  $1,618,229 

Then,  multiplying  this  figure  by  the  95%  confidence  level  factor  of  1.96  gives  $3,171,729. 
Lastly,  we  dvide  $3,171,729  by  the  estimated  total  hit  value  $67,130,830  and  find  an 
actual  precision  of  ±  4.7247%. 

While  4.7247%  may  seem  only  slightly  higher  than  3%,  in  actuality  4.7247  is 
57.5%  greater  than  3%.  Additionally,  the  range  allowed  by  the  auditors  was  from 
$66,198,510  to  $70,293,264.  However,  the  range  calculated  here  was  fi-om  $63,959,100 
to  $70,302,560.  This  range  is  over  $2,000,000  greater  than  that  claimed  by  the  DODIG  in 
the  audit  report. 
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E.  DIFFERENCES 


1.  Universe  of  S100,000purdiafcreqaciti 

While  reviewing  the  background  information  for  the  data  that  was  published  in 
the  final  audit  report,  several  discrepasdes  arose  in  the  supportii^  data.  The  first 
category  that  dififm  between  the  report  and  all  supporting  woildi^  papers  is  the  dollar 
value  of  the  univase  of  items  which  include  purchase  requests  of  $100,000  or  more. 
While  there  was  an  initial  figure  for  the  entire  universe  value  ($691. 1  million),  each  of  the 
four  individual  values  was  obtained  throu^  projections  fix>m  the  sample  of  107  items. 
Although  the  four  sample  values  were  r^nlariy  being  checked,  re-checked,  and  changed 
during  the  audit,  the  four  figures  listed  in  Table  XIV  were  ^patently  never  re-totaled  after 
the  projections  were  made  to  verify  that  they  matched  the  original  universe  value.  All 
four  are  listed  in  the  working  paper  summary  with  the  dollar  figures  seen  in  Table  XTV. 
This  $3  million  difierence  is  not  significant  but  its  impact  carries  forward  to  other 
categories,  as  seen  in  the  next  section. 


Table  XIV.  $100,00 

0  UNIVERSE  BREAKDOWN 

Value 

Adjusted 

$170,972,701 

Undeterminable 

69,543,164 

Excessive 

156,563,299 

Reasonable 

297.062.229 

Total 

$694,141,393 

2.  Adjusted  Unhrene 

The  second  major  difference  discovered  was  the  total  dollar  value  of  the  adjusted 
universe.  In  all  the  major  conclusions,  there  were  two  different  dollar  sizes  for  the 
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adjusted  universe;  $520.2  million,  and  $523 .2  million.  This  difference  in  adjusted 
universe  value  ($3  million)  is  derived  from  figures  taken  from  the  rq)oit.  As  seen  in  Table 
XrV,  the  universe  value  was  $694,141,393.  When  the  adjustment  ($170,972,701)  is 
subtracted,  the  adjusted  universe  value  of  $523,168,692  is  obtained  (see  Colunm  (1)  of 
Table  VII).  Since  the  adjusted  universe  was  not  actually  used  in  the  audit  estimate,  this 
difference  is  of  no  practical  significance. 

3.  ASO  Universe 

The  next  notable  difference  was  the  total  dollar  value  for  the  889  ASO  purchase 
requests  over  $100,000.  When  projections  were  made  for  the  three  largest  categories 
(undeterminable,  reasonable,  and  excesnve),  the  working  pqrer  cover  sheets  correctly 
listed  the  ASO  total  as  $495,886,170  (see  Appendix  B),  however  the  detailed  working 
papers,  used  to  calculate  the  projections,  totaled  $498,856,171  (see  column  (4)  of  Table 
K). 


Table  XV.  ADJUSTED  UNIVERSE  CORRECTIONS 


Original  Value  Corrected  Value 


Undetermirud)le 


Reasonable 


Excessive 


297,062,229 


156.563.299 

155.948.459 

$523,168,692. 

$521,102,679 

Table  XVI.  EXCESSIVE  CORRECTIONS 
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Table  Xm  UNNECESSARY  CORRECnCWS 


Original  Value 

Corrected  Value 

Dafercnce 

Eumi 

$68,245,887 

$67,987,400 

v.'-v- .  - 

Other 

$78,753,456 

The  specific  mistakes  were  a  $30,000  shoitage  fisr  ASO  stratum  2  and  a 
$3,000,000  overage  Sor  ASO  stratum  4.  Tables  XV  throu^  XVII  give  the  fectm^Kited 
projectitms  for  the  cat^ories  originally  listed  in  Table  Vn,  Summary  of  Audit  Results. 
While  the  origiiul  errors  appeared  significant  (a  $2.97  million  total  difference),  the  true 
bottom  line  inqiact  to  be  investigated  is  the  change  in  the  projected  monetary  benefits.  As 
Table  XVn  shows,  this  amounted  to  $258,487  which  is  less  than  the  average  value  of  otM 
purchase  request  fi-om  the  original  universe  ($691,171,343  +  1430  $483,337)  and  only 

.38%  (258,487  -i-  68,245,887)  of  the  original  projected  monetary  benefits. 

4.  Excessive 

The  firud  notable  difference  that  merits  attention  is  the  total  value  of  excessive 
purchase  requests.  As  Table  Vn  shows,  ^^len  the  total  for  the  adjusted  universe  is 
calculated,  the  dollar  value  for  excessive  purchase  requests  is  $156,563,299.  But  ^^dien 
the  components  of  the  excessive  category  are  shown,  they  total  $156,923,374.  -  a 
$360,075  difference. 

The  best  explanation  for  this  diffoence  is  amilar  to  that  described  in  Section 
E.l  of  this  chapter.  Again,  it  appears  to  be  an  instance  ^^lere  one  set  of  figures  was  used 
to  make  the  projection  of  the  excessive  value.  Then,  during  a  separate  poim  in  the  audit, 
the  three  components  of  the  excesave  value  were  projected.  Apparently,  the  three 
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con^ponents  were  never  totaled  to  ensure  they  urnmed  to  die  lepOTtely  projected 
excessive  total. 

5.  SunuBuuy 

During  the  review  of  all  figures  behind  the  audit  report,  it  became  dear  that  die 
figures  eventually  were  d)tained  in  pieces.  Retracing  the  i»ocess  shows  that  die 
condusitHis  obtained  were  baaed  on  shifting  fiiundations.  The  moat  imp(»tant  figures  are 
those  used  to  make  projections.  As  seen  in  Section  E.3,  $3  million  and  $30  thousand 
e^^ors  are  the  basis  for  completely  recalculating  the  projected  monetary  benefits  of  the 
entire  audit.  While  the  ultimate  resuh  may  not  be  too  much  different,  these  corrections 
create  a  basis  fisr  questioning  all  assumptions  made  during  the  audit. 
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V.  SUMMARY,  CONCLUSIONS  AND  RECXNMMENDATIONS 


A.  SUMMARY 

The  objective  of  this  tiiesU  was  to  examme  how  die  Dqiartment  of  Defense 
Inspector  General  (DODIG)  tncoiporatod  stratified  — »npHne  when  uiditiBg  inventcwy 
control  ptnnts.  The  primary  fixus  was  on  how  the  DODIG  used  stratified  Mmpitng  to 
project  the  potential  monetary  benefits  of  their  findingit  A  secondary  issue  was  to 
examine  the  stated  confidence  levd  and  predskm  of  their  ccmchisUms  to  see  if  they  were 
as  tight  as  stated  in  the  final  rqwrt. 

Charter  n  introduced  the  methodology  for  using  stratified  sampling  to 
projections  accurately.  ¥or  stratified  samiding  to  be  bes«ficial  to  the  auditors,  strata 
should  be  more  homogeneous  than  the  entire  sanqile  in  order  to  reduce  variability  and 
reduce  the  total  required  sanqile  size.  Chapter  II  distinguished  between  frequency 
estimation  and  dollar  value  estimation,  both  of  vdiich  were  used  during  the  DODIG  audit. 
The  chapter  also  included  a  brief  discussion  of  confidrace  intervals  and  tlwir  use  in  stating 
concluaons  about  a  sample  mean.  The  final  section  of  Chapter  n  was  a  description  of  the 
term  “stratification”  in  inventory  rnanagonem.  Stratification  was  critical  for  NAVSUP 
when  reporting  the  impact  of  the  audit.  NAVSUP  used  three  figures  when  calculating  the 
audit’s  impact.  One  of  the  figures,  the  DILS  rate,  was  determined  from  reviewing  the 
“STRAP’  while  the  audit  was  in  process. 

The  data  that  was  coUected  and  used  throughout  the  audit  was  described  in 
Chapter  m.  Appendices  B  through  M  include  the  detailed  data  that  was  described  in 
Chapter  in.  The  beginning  of  Chapter  m  included  a  description  of  how  the  auditors 
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began  the  audh  by  narrowing  the  scope,  choosing  their  san^  size  and  then  divkhog  their 
sample  into  the  four  strata.  The  following  section  briefly  mentumed  some  of  the  ttidhore’ 
various  considerations  n^ien  cat^orizii^  their  findings.  Charter  m  then  fmsented  the 
categorization  of  data  that  ultimately  led  to  the  DODIG’s  most  inqmrtant  finding  >  the 
projected  monetary  benefits  of  unnecessary  purchase  requests  ($68.2  ndllion).  lunally,  the 
chiq)ter  concluded  with  NAVSUP’s  use  of  the  audit  results  to  rqwrt  the  budgetary  infract 
for  the  following  fiscal  year. 

Chapter  IV  presented  an  analytis  of  the  data  coUected  to  itudce  the  audit 
conclutions.  The  first  part  of  the  chiqrter  discussed  the  way  the  DODIG  selected  the 
“adjusted”  universe,  sample  size  and  strata  and  their  stated  confidence  level.  The  next 
section  is  the  author’s  analysis  of  the  sanq}le  size  dedaons  and  the  author’s  investigation 
of  alternate  methods  of  selecting  a  sample  size  and  stratifying  the  chosen  sample.  The 
chapter  also  presented  and  computed  an  ^gregate  estimated  confidence  imerval  based  on 
the  stratifications  made  during  the  audit  and  compared  it  to  the  stated  accuracy  of  the 
DODIG  report.  The  next  section  reviewed  differing  dollar  figures  used  to  make 
projections.  In  particular,  this  section  explained  how  numbers  changed  as  the  audit 
progressed  and  how  apparently  no  thorough  final  review  was  conducted  to  ensure  that 
conclusions  made  early  in  the  audit  were  based  on  the  same  figures  as  those  used  to  make 
conclusions  at  the  end  of  the  audit. 

B.  CONCLUSIONS 

During  the  DODIG  audit ,  audited  commands  naturally  put  their  emphasis  on  the 
individual  findings.  Each  finding  is  closely  scrutinized,  verified,  and  argued  over  before 


43 


the  auditors  are  allowed  to  rqwrt  their  resuh.  What  is  dearly  more  difficult  to  argue 
about  are  the  iMojections  made  after  the  ttidh  ftndiiigs  are  gathered.  Even  if  the 
projections  are  explained,  the  precision  of  the  findii^  is  often  beytmd  sinqde 
imerpretations.  Therefore,  the  auditors’  repent  results  tend  to  be  the  final  wmd.  The 
precision  of  the  audit  projectiems  are  critical  because  the  audited  cemimands  can  then  more 
accurately  interpret  the  audit  findings. 

During  the  process  of  reviewing  the  audit,  results  were  found  to  be  based  on 
projections  fix>m  the  chosen  sample.  However,  one  of  the  unusual  condusions  fit>m  the 
sample  was  that  the  universe  dze  should  be  adjusted  based  on  21  sampled  items  valued  at 
S41.1  million  that  were  determined  to  be  outside  the  scope  of  the  audit.  The  DODIG 
auditors  projected  that  374  items  totaling  $171  million  should  be  removed  fi’om  the 
original  universe.  The  result  was  an  unorthodox  “adjusted  universe”  of  1056  items  with 
an  estimated  value  totaling  SS20.2  million.  However,  all  subsequem  calculations  and 
estimates  were  based  on  the  original  population  size  of  1430.  Because  an  accurate 
universe  is  essential  for  all  sampling,  the  original  universe  should  have  been  completdy 
reviewed  to  remove  items  that  were  not  to  be  induded  in  the  audit  and  the  resulting 
universe  size  used  in  all  the  subsequent  analyses. 

Another  finding  of  the  audit  review  was  that  there  was  no  documented  procedure 
to  explain  why  a  sample  size  of  107  was  chosen.  Conversations  with  several  of  the 
DODIG  statistidans  revealed  that  no  one  could  recall  how  a  sample  size  of  107  was 
chosen.  Despite  the  time  between  the  audit  and  this  thesis,  107  should  have  been 
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voifiable.  Thi$  v^Mild  have  been  possible  if  the  procedure  were  documented,  Hdrich  it 
clearly  was  not.  This  issue  is  fiirther  discussed  in  the  rectmunendations. 

A  significant  finding  of  this  thesis  be  fimnd  in  A{^)codioes  B  and  M.  These 
q)pendices  show  that  63.7%  of  the  potentuU  monetary  benefits  of  unnecessary  purchase 
requests  were  derived  fiiom  the  sanqrle  of  the  fourth  stratum.  That  sanqrle  included  only 
3.4%  (44  -t-  1301)  of  the  stratum  universe  of  purchase  requests  and  only  3.0% 
(10,736,025  +  358,632,509)  of  the  stratum  universe’s  value.  However,  this  same  stratum 
included  91%  of  all  purchase  requests  and  nearly  52%  of  the  purchase  request  value  for 
the  entire  universe.  In  light  of  the  very  small  sanqrle  used  in  the  fixirth  stratum,  the 
OODIG  claim  that  their  findings  are  based  on  a  95%  confidence  levd  with  a  stated  3% 
level  of  precision  is  definitely  opoi  to  question. 

A  related  conclusion  of  this  thesis  is  that  there  was  an  arbitrary  decision  to  divide 
the  strata  to  ensure  100%  inclusion  of  the  highest  dollar  value  purchase  requests,  80%  of 
the  second  highest  strata,  30%  of  the  next  higher  strata,  and  then  pulling  the  remaining 
j„.aples  fi’om  the  lowest  dollar  value  strata.  This  focus  on  the  dollar  value  of  purchases 
created  unrealistic  assumptioru  about  the  ruimber  of  line  hems  considered  unnecessary. 
Specifically,  the  DODIG’s  projected  number  of  unnecessary  purchase  requests  was  399 
(see  i^pendix  M).  This  conclusion  assumes  that  neariy  28%  (399  -i-  1430)  of  all  ICP 
purchase  requests  were  at  least  partially  utmecessary.  While  this  figure  can  be  stated  with 
95%  confidence,  the  wide  range  (e.g.,  lack  of  precirion)  of  this  interval  must  also  be 
included  when  reporting  such  numbers.  Instead,  no  precision  was  given  for  the  number  of 
utmecessary  purchase  requests. 
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A  nu^or  finding  of  this  thesis  resulted  from  a  test  of  the  stated  audit  confidence 
level  and  precision  of  the  dollar  value  estimates.  Wlule  the  audit  results  were  said  to  have 
a  precision  of  ±  3%  for  the  dollar  value,  the  actual  precision  determined  was  ±  4.7%  or 
about  S7%  and  $2  million  larger  than  stated.  This  is  in4)ortant  because  all  the  auditors’ 
work  was  supposed  to  have  led  to  a  95%  confidence  levd  and  ±  3%  precision.  The 
statistical  accuracy  of  the  DODIG  fimlings  is  ahical  to  anyone  reviewing  the  rqrait. 
Without  this  accuracy,  the  findings  are  continually  open  to  questions. 

C.  RECOMMENDATIONS 

This  thesis  has  highlighted  several  potential  opportunities  for  improving  the 
outcome  of  DODIG  audits  and  the  confidence  in  the  findings  rendered. 

1.  DODIG  Should  Develop  Standard  Statistical  Procedures  and  Publish 
Them 

During  conversations  with  several  of  the  DODIG  representatives,  it  became 
clear  that  audits  such  as  those  conducted  on  the  Navy  ICPs  will  be  occurring  with 
increased  frequency.  Naturally,  the  audited  commands  are  sensitive  to  findings  that  put 
them  in  a  negative  light.  This  creates  an  incentive  for  the  audited  commands  to  criticize 
the  way  the  auditors  make  their  findings  and  conclusions.  In  order  to  avoid  other  reviews 
like  this  thesis,  and  also  to  simplify  the  process  for  the  auditors,  standardized  procedures 
of  statistical  steps  should  be  developed  and  eventually  published.  This  method  of  full 
disclosure  on  the  part  of  the  auditors  would  benefit  everyone  in  the  audit  process. 
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2.  A  Suoxinct  Sttmnuuy  of  the  Amlit  Fudiap  Should  Be  Provided  to 


Audited  Commaads 

A  thorough  review  of  all  the  DODIG  working  papers  and  the  final  audit  rqwrt 
clearly  showed  that,  during  the  course  of  the  audit,  conclusions  were  readied  and  then 
changed  because  some  of  the  early  assumptions  were  changed.  The  resuh  was  that 
conclusions  reached  during  various  phases  of  the  audit  were  based  on  differing 
assun^tions.  For  example,  the  projections  made  for  the  SPCC  portion  of  the  premature 
purchase  requests  (Appendix  K)  were  ineiqilicably  made  uang  an  earlier  resuh  of  an 
“excessive "  value  projection.  However,  throughout  the  rest  of  the  audit,  projections  for 
all  other  groups  were  made  using  the  original  universe  value.  This  small  inconsistency 
created  a  difference  in  the  two  totals  for  “excessive”  given  in  Columns  (1)  and  (2)  of 
Table  vn.  By  requesting  a  summary  of  the  conclusions,  it  would  be  easy  for  the  audited 
commands  to  verify  that  all  audh  findings  wo-e  based  on  the  same  set  of  facts. 

3.  Statistical  Ranges  Should  Be  Provided  to  Audited  Commands  With 

Supporting  Data 

When  commands  are  audited,  they  reedve  the  audh  results  with  a  statement 
that  the  auditors  have  95%  confidence  in  their  findings  within  a  certain  range  or  precision. 
This  is  normally  a  given  that  is  not  open  to  question.  However,  this  critical  figure  should 
be  fully  explained.  When  only  3%  of  one  stratum  is  reviewed  and  the  audit  findings  fi^om 
this  stratum  are  used  to  extrapolate  63.7%  of  the  findings,  then  h  is  essential  to  the 
audited  commands  that  they  have  a  complete  understanding  of  why  that  was  done.  As 
mentioned  earlier,  the  range  of  the  audit  findings  are  as  important  as  the  findings 
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themselves.  Therefore,  whoi  a  statenMiit  is  made  about  confidence  level  and  V* 
precision,  these  should  be  provided  with  the  dollar  values  to  which  they  refer.  Then 
figures  can  be  verified  through  the  use  of  procedures  like  those  used  in  Cht^ner  fV.D. 
When  this  process  is  complete,  a  clear  calculation  of  the  precision  is  then  available  for 
review. 

4.  Validate  Sample  Sizes  After  the  Audit 

This  thesis  provides  a  process  that  can  be  used  to  determine  the  precision  of 
selected  sample  sizes.  For  commands  that  are  bong  audited  for  attributes  only  (umike  the 
ICPs),  Chapter  IV.C  provides  a  process  to  verify  the  accuracy  of  the  auditor’s  precision. 
Additionally,  commands  being  audited  for  both  variables  and  attributes  can  also  use  the 
process  to  ensure  the  sample  size  is  statistically  valid.  If  the  audited  command  concludes 
the  auditor’s  stated  precision  is  inaccurate,  the  auditors  should  be  approached  immediately 
to  allow  them  an  opportunity  to  defend  their  findings  or  issue  an  addendum  to  the  findings 
stated  in  the  audit  report. 

5.  Additional  Research  is  Needed 

The  primary  purpose  of  this  thesis  was  to  review  the  DODIG’s  sampling  plan 
which  was  used  in  the  evaluation  of  the  two  ICPs’  purchase  requests.  What  was  not 
considered,  but  appeared  to  be  an  area  of  serious  debate  during  the  audit,  were  the  issues 
of  what  constitute  premature,  unnecessary,  or  insufBcient  purchases.  A  major  factor  in 
this  determination  was  the  definition  of  “stockage  objective.”  In  order  to  narrow  the  sdjpe 
of  this  thesis,  these  terms  were  not  debated.  However,  correspondence  between 
NAVSUP  and  the  DODIG  clearly  showed  that  the  definitions  were  open  to  interpretation. 
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A  iiK>re  tn-dq>th  look  into  this  area  would  benefit  the  ICPs  and  others  depoident  oa 
stockage  objectives  to  e£fectivdy  nuuuige  warehouse  inventcMies. 
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GLOSSARY 


Approved  Acquisition  Objective  (AAOV  Tbe  quantity  of  an  hem  authoriased  for 
peacetime  a^  wartime  requirements  to  equip  ai^  sustain  U.S.  and  Allied  Forces  in 
accordance  with  current  DOD  pdides  and  phms. 

Continycnev  Retention  Stock  fCRSV  That  portion  of  the  quantity  of  an  hem  greater  than 
the  AAO  and  economic  retention  stodc  {or  wfaidi  there  is  no  i»edictabie  or 

quantifiable  requirement,  aiul  that  would  normally  be  allocated  as  Potential 
Reutilization  Stock  ,  excq>t  for  a  determination  that  tte  quantity  will  be  retained  for 
specific  contingendes. 

Due  In  Long  Supply:  Assets  expected  to  arrive  into  the  \^lesale  supply  system  in  excess 
of  all  known  or  expected  requirements  during  some  time  period,  usually  thought  of  as 
in  excess  of  the  Retention  Limh. 

Economic  Retention  Stock  (ERSV  That  portion  of  the  quantity  of  an  hem  greater  than 
the  AAO  determined  to  be  more  economical  to  retain  for  fut^  peacetime  issues  than 
to  dispose  and  satisfy  projected  future  requirements  through  new  procurement  and/or 
repair.  To  warrant  economic  retention,  an  hem  must  have  a  reasonably  predictable 
demand  rate. 

Excessive:  Purchase  requests  that  fidl  into  other  the  “pronature”  or  “utuecessary” 
categories  described  below. 

Hardware  Systems  Command:  A  headquarters  activity  that  is  responsible  for  the 
procurement  and  technical  support  of  weapons  system  requirements  (e.g..  Naval  Sea 
Systems  Command  and  Naval  Air  Systems  Command). 

Non-pro|ectable:  A  purchase  request  that  could  not  be  used  to  predict  overall  error 
because  the  ICPs  had  taken  independent  action  to  reduce  occessive  purchases.  The 
actions  woe  taken  because  of  ICP  item  manago  or  supervisory  review  or  an  HSC  had 
directed  a  curtailment  to  the  purchase. 

Potential  Reutilization  Stock:  Material  identified  by  an  item  numago  for  possible  disposal 
but  with  potential  for  reutilization;  or  material  that  has  the  potential  for  being  sent  by 
an  item  manago  to  the  Defense  Reutilization  and  Marketing  Service  for  possible 
reutilization  by  anotho  Component  or  by  a  Federal,  state,  or  local  government  agency. 

Premature:  A  purchase  request  with  a  quantity  that  exceeds  the  stockage  objective  by 
more  than  twelve  months  of  forecasted  requirements. 


SO 


Reasonable:  Purchase  requests  that  do  not  fit  into  the  “excessive”  or  “undeterminable” 
definitions  described  here. 


1  Limit:  The  maximum  quantity  of  on-hand  material  that  may  be  retained  in 
stock.  This  quantity  is  deter.tun^  by  summing  AAO,  ERS,  and  CRS. 


Safety  Level:  The  value  of  the  expected  net  inveotoiy  (on  hand  minus  backorders)  just 
before  an  order  arrives.  This  is  a  “cushion”  of  sto^  which  is  kept  on  hand  to  cover 
variations  in  demand  during  lead  time. 


Undeterminable:  A  purchase  request  for  whidi  DODIG  was  unable  to  determine  the 
reasonableness  because  the  ICPs  could  not  provide  verifiable  requirements  data  as  of 
September  30, 1990  or  requirements  were  dependent  on  a  critical  management  decision 
by  a  hardware  systems  command  (HSC). 


Unnecessary:  A  purchase  request  with  a  quantity  in  excess  of  five  years  of  forecasted 
requirements. 
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APPENDIX  A:  NOTATION  AND  EQUATIONS  USED  IN 
STRATIFIED  SAMPLING 


Notations  used  in  Stratified  Sanqdiog: 

vahw  of  the  “rth  observation 
N  total  number  of  units  in  aU  strata  (the  population) 
c  -  coeflBicieiit  of  variation 
s  -  standard  deviation  of  a  san^e 
T  -  the  mean  of  the  x,  observations 

L  -  number  of  strata  into  which  the  population  is  divided  for  purposes  of  stratified 
sampling 

p  =  population  proportion  that  possesses  a  specified  charactoristic 
p^-m  unbiased  estimator  of  the  poinilation  proportion  p 

-  the  population  proportion  in  stratum  “i”  that  possesses  a  specified  charactoistic 
p  =  an  unbiased  estinutor  of  Pf 

-  total  number  of  units  in  stratum  “i” 

n  =  number  of  units  sampled  out  of  the  population 
/i.  =  number  of  units  sampled  in  stratum  T’ 

V  =  estimated  variance 

q=l.p 

<It  =  1-  Pi 
i  =  stratum  index 

=  sample  mean  of  the  /th  stratum 
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»  estimate  of  population  mean  (F ) 

E  »  standard  error 

z  » the  normal  deviate  related  to  the  confidence  level  (e.g.,  l.%  for  95%  confidence) 
mean  estimator 

(1)  coefficient  of  variation,  a  measure  of  rdative  variability:  (Neter,  1993,  p.  84) 

c=.00(±) 


(2)  estimator  of  the  population  proportion  p:  (Mendenhall,  1990,  p.  117) 

p.-jjp.p. 


(3)  estimated  variance  of  p^:  (Mendenhall,  1990,  p.  117) 


(4)  bound  on  the  error  of  estimation:  (Mendenhall,  1990,  p.  1 17) 


(5)  standard  error  of  a  percent  for  a  sample  drawn  fix>m  a  finite  population:  (EQll,  1962, 
p.  A-l) 


±E 
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(6)  vriaacc  ci  i  saiqirie:  (Nder,  1993.  p.  t2) 


±(x,-n 


M 

(8)  estimated  mean  of  the  pt^Hilation;  (Riy,  1972.  p.  S3) 

Y-itw. 

^  tel 


(9)  variance  of  the  estimator  x  :  (Riy.  1972.  p.  S3) 
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APPENDIX  B:  SUMMAIIY  OF  UMVEflSE  AND  SAMPLE  VALUES 


C:  COMPtITg  UST  OF  SAMPtEP  ITEMS  CINITIALLY) 


Numbart 


1|NA1<01 


2  NA1-02 


3  NA1-03 


NA1-04 


5  NA1-05 


6  NA1-06 


NA1-07 


NA1-08 


NA2-01 


10  NA2-02 


11  NA2-03 


12  NA2-04 


13  NA2-0S 


14  NA2-06 


15  NA2-07 


16  NA2-08 


17  NA2-09 


18  NA2-10 


19  riA2-11 


20  NA2-12 


21  NA2-13 


22  NA2-14 


23  NA2-1S 


24NA2-16 


25  NA2>17 


26  NA2>18 


27  NA2*19 


28  NA2-20 


29 

NA3-01 

30 

NA3-02 

31 

NA3-03 

32 

NA3-04 

33 

NA3-05 

34 

NA3-06 

35 

NA3-07 

36 

NA3-08 

37 

NA3-09 

38 

NA3-10 

39 

NA3-1 1 

40 

NA3-12 

41 

NA3-13 

42 

NA3-14 

43 

NA3-15 

44 

NA3-16 

45 

NA3-17 

Extandad  Vahia 


413.666.416 


11,967.280 


11,104,740 


9,006,800 


7.935.475 


2.902.650 


3,480,000 


3,112,110 


3,893,507 


3,035,373 


2,774,636 


2,602,380 


2,629,424 


4.916,280 


2,701,218 


3,141,600 


3,655,000 


2,569,869 


3,170,952 


3,272,160 


2.549,261 


2,827,366 


3,228,039 


3,007,680 


4,690,796 


1,486,694 


1,241,267 


1,314,587 


1,269,709 


1,821,430 


1,673,360 


1,122,028 


2,340,030 


1,346,856 


1,141,950 


1,067,950 


1,146,958 


1,334,520 


1,159.297 


1,994,278 


1,254,300 


2,023,104 


24,738,318 


APPENDIX  C:  COMPLETE  UST  OF 


ITEMS  (mmAUY) 


NA4>01 


47  NA4>02 


48  NA4-03 


49  NA4-04 


50  NA4-0S 


51  NA4-06 


52  NA4-07 


53  NA4-08 


54  NA4-09 


NA4-10 


56  NA4-11 


57  NA4-12 


NA4-13 


59  NA4-14 


60  NA4-15 


61  NA4-16 


62  NA4-17 


NA4-18 


64  NA4-19 


65  NA4-20 


NA4-21 


67  NA4-22 


68  NA4-23 


NA4-24 


70 

NS1-01 

71 

NS1-02 

72 

NS2-01 

73 

NS2-02 

74 

NS2-03 

75 

NS2-04 

76 

NS2-05 

77 

NS2-06 

78 

NS3-01 

79 

NS3-02 

80 

NS3-03 

81 

NS3-04 

82 

NS3-05 

83 

NS3-06 

84 

NS3-07 

85 

NS3-08 

86 

NS3-09 

87 

NS3-10 

NS4-01 

89 

NS4-02 

90 

NS4-03 

411.350 


126,943 


587.070 


298,580 


163,896 


180,000 


225,146 


117,539 


128.370 


227,125 


294,000 


434,968 


113,920 


313,200 


121,726 


113,971 


185,786 


340,000 


438,070 


106,587 


130,314 


186,480 


203,274 


10,186,600 


6,614,400 


3,840,000 


3,592,000 


3,150,000 


3,084,156 


3,045,690 


2,665,391 


1,750,000 


1,602,685 


1,108,500 


1,150,000 


1,152,000 


1.102.000 


2,232,510 


2,100,000 


2,256,012 


1,040,000 


225,215 


350,980 


145,000 
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APPENDIX  C:  COMPUETE  UST  OP  8AMPU0  ITBy»  (MmAUVI 


91 

NS4^ 

214,773 

92 

NS4-05 

155,682 

EE 

NS4-06 

109,152 

NS4-07 

EE 

NS4-08 

610,977 

EE 

NS4-09 

266,489 

NS4-10 

144,750 

ee 

NS4-11 

268,935 

99 

NS4-12 

170,880 

iEE 

NS4-13 

105,934 

NS4-14 

125,844 

NS4-15 

291,816 

103 

NS4-16 

731,766 

104 

NS4-17 

278,694 

NS4-18 

129,422 

NS4-19 

153,920 

107 

NS4-20 

125,022 

5,087.281 

$229,399,582 

*  A  =  ASO 

S  >  SPCC 

Ist  Number  »  Stratum 

2nd  and  3rd  Numbers  « 

Mumbar  Within  Stratum  I 
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APPENDIX  E:  COMPLETE  UST  OF  SAMPLED  ITEMS  (REVISED) 


Samoie  Numbers  I  National  Stock  Numbers 


EQI 


m 

\m 

\m 

\m 


Im 

jcTH 


lES 
I  EE] 


lES 

m 

S 

lEEll 

\m 


NA1-01 


NA1-02 


NA1-03 


NA1-04 


NA1-06 


NA1-07 


NA1-08 


NA2-01 


NA2-02 


NA2-04 


NA2-05 


NA2-06 


NA2-07 


NA2-08 


NA2-10 


NA2-14 


NA2-15 


NA2-16 


NA2-17 


NA2-18 


NA2-19 


NA2-20 


NA3-01 


NA3-02 


NA3-03 


NA3-04 


NA3-05 


NA3-06 


NA3-07 


NA3-08 


NA3-10 


NA3-1 1 


NA3-12 


NA3-13 


NA3-14 


NA3-15 


NA3-17 


NA4-01 


NA4-02 


NA4-03 


NA4-05 


NA4-07 


NA4-08 


NA4-10 


1615-01-201-9639 


f 660-01 -277-8238 


2840-01-251-7227 


5960-LL-NEE-A190 


2840-LL-WY5-5024 


6605-01-054-3776 


1615-01-201-9601 


1615-01-158-9678 


1650-01-161-4420 


2840-01-142-8818 


1560-01-155-7014 


5998-01-306-1972 


4920-01-124-9246 


4920-01-220-4520 


5865-01-196-9869 


1430-01-325-2512 


4920-01-279-8220 


5855-01-052-6849 


1620-00-761-4903 


4920-01-156-1393 


1270-01-256-8264 


4920-01-124-9245 


1620-01 


5895-01 


1560-01 


1740-01 


5841-01 


2835-01 


2925-01 


1630-01 


1730-01 


6720-01 


7021-01 


6605-01 


6605-01 


5963-01 


1280-01 


-177-1891 


-303-7755 


-300-7768 


-062-1657 


-004-7531 


-256-8378 


-277-3508 


-106-4900 


-126-6239 


-181-5872 


-283-3749 


-245-8209 


-027-4172 


-154-2794 


-095-2982 


6610-01-278 


1560-01-284 


6615-01-183 


5999-01-271 


4920-01-251 


1610-00-887 


2840-01-150 


9291 


5093 


7413 


1243 


7174 


0392 


6734 


Quanti 


94 


74 


51 


253 


16 


61 


Extendad  Value 


$13,666,416 


11,967,280 


11,104,740 


10,634,238 


7,935,474 


7,381,000 


6,397,046 


2,902,650 


3,480,000 


3,893,507 


3,035,373 


2,774,636 


2,602,380 


2,629,424 


2,701,218 


3,170,952 


3,272,160 


2,549,261 


2,827,366 


3,228,039 


3,007,680 


4,690,796 


1,486,694 


1,241,267 


1,314,587 


1,269,709 


1,821,430 


1,673,360 


1,122,028 


2,340,030 


1,141,950 


1,067,950 


1,146,958 


1,334,520 


1,159,297 


1,994,278 


2,023,104 


411,350 


126,943 


587,070 


163,895 


180,000 


225,146 


128,370 


227,125 


Subtotals 


$69,086,194 


46,765,442 


22,137,162 
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APPENDIX  E:  COMPLETE  UST  OF  SAMPLED  ITEMS  (REVISED) 


\m 


NA4-12 


NA4-15 


NA4-16 


NA4-18 


NA4-19 


NA4-21 


NA4-22 


NA4-23 


NA4-24 


NS2-01 


NS2-02 


NS2-03 


NS2-04 


NS2-05 


NS2-06 


NS3-01 


NS3-02 


NS3-03 


NS3-04 


NS3-05 


NS3-06 


NS3-07 


NS3-08 


NS3-09 


NS3-10 


NS4-01 


NS4-02 


NS4-03 


NS4-04 


NS4-06 


NS4-07 


NS4-08 


NS4-10 


NS4-12 


NS4-13 


NS4-14 


NS4-15 


NS4-17 


NS4-19 


NS4-20 


4810-01-271-8852 


6615-01-129-8410 


1280-01-186-1434 


1680-01-242-9698 


2840-01-281-3618 


7050-01-098-5523 


1680-01-159-9153 


6680-01-175-9116 


1620-01-158-5958 


5895-01 


2010-01 


5960-01 


6625-01 


6625-01 


6625-01 


-281-2401 


-111-9593 


-302-4456 


-234-0485 


-146-1554 


-233-7104 


6150-01-306-7242 


1420-01-108-5915 


5820-01-020-2762 


4310-01-187-5041 


5999-00-619-7838 


4320-01-220-1747 


6625-01-258-3140 


5845-01-307-6466 


6625-01-259-7355 


6695-01-299-8473 


6130-01-155-2338 


5820-00-334-8407 


5999-01-255-1816 


2825-00-371-7899 


6605-01-030-0004 


2010-01-222-5283 


5985-01-119-3998 


4820-01-090-6529 


5915-00-527-9524 


4320-01-062-1473 


2010-01-144-2462 


6625-01-268-6800 


2865-01-164-1509 


2990-01-134-6899 


5998-01-183-7818 


S  =  SPCC 


1st  Number  Stratum 


2nd  and  3rd  Numbers  =  Number  Within  Stratum 


294.000 


313,200 


121.726 


185,786 


340,000 


106,587 


130,314 


186,480 


203,274 


3,840,000 


5,388,000 


3,150,000 


3,084,156 


3,045,690 


2,665,391 


1,750,000 


1,602,685 


1,108,500 


1,150,000 


1,152,000 


1,102,000 


2,232,510 


2,100,000 


2,256,012 


1,040,000 


225,215 


350,980 


110,000 


214,773 


109,152 


482,030 


226,288 


144,750 


1 70,880 


105,934 


125,844 


291,816 


278,694 


153,921 


125,022 


3,931,266 


mLESMl 


21,173,237 


15,493,707 


3,115,299 


$188,316,707 


APPENDIX  F:  SUMMARY  OF  ADJUSTED  UNIVERSE  AND  SAMPLE  VALUES 
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lASO 


Stratum 

I _ ^ 

_ 2_ 

_ 3_ 

4 


ISPCC 


Stratum 

_ 1_ 

_ 2_ 

_ 3_ 

4 


[TOTAL 


Stratum 

1 

2 

3 

4 


APPENDIX  H:  REASONABLE  PURCHASE  REQUESTS 


lASO 

_ 

_ 

. . ~""l 

■■■■ 

IHIHii 

hhbh^i 

Stratum 

Hits 

HitVakia 

Projected 

Hits 

Projected  Value 

1 

3 

♦31,007.495 

3 

$31,007,495 

2 

6 

25.869,400 

8 

33,814,662 

3 

5 

10,827.385 

19 

43.638.953 

4 

5 

2.376,480 

164 

99.788.676 

19 

$70,080,760 

195 

$208,249,786 

SPCC 

Hits 

Hit  Value 

Projected 

Hits 

Projected  Velue 

1 

1 

$614,400 

1 

$6,614,400 

2 

5 

17,717,238 

5 

17,717.238 

3 

6 

11,560,810 

13 

25.866,326 

4 

7 

1.578.600 

179 

38,614.479 

19 

$31,471,048 

198 

$88,812,443 

TOTAL 

_ 

IHHIH 

Stratum 

Hits 

Hit  Value 

Projected 

Hits 

Projected  Value 

1 

4 

$31,621,895 

4 

$37,621,895 

2 

11 

43,586,638 

13 

51,531,900 

3 

11 

22,388,195 

32 

69,505,279 

4 

12 

$71,659,360 

343 

138.403.155 

38 

$169,256,088 

392 
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APPENDIX  I:  EXCESSIVE  PURCHASE  REQUESTS 


Hits 

Projected 

Hits 

Projected  Value 

2 

«20,063,461 

2 

$20,063,461 

6 

11,778,245 

8 

14,334,254 

11 

10,596,389 

35 

35,656,225 

18 

2,701,732 

534 

86,509,359 

37 

$45,139,827 

579 
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APPENDIX  J:  NON-PROJECTABUE  PURCHASE  REQUESTS 


APPENDIX  K:  PREMATURE 


REQUESTS 
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APfENOIXL:  UNNECESSAIIY  nMCHASE  RfQUESTS 
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UNKNOWN  180%!.  CONPneiCE  LfVEL  98% 


±  2% 

±  3% 

±  4% 

±  5% 

200 

300 

400 

196 

500 

■ 

217 

1.000 

375 

278 

1,500 

429 

306 

2,000 

■EESi 

462 

322 

2,500 

1225 

748 

484 

333 

3,000 

1334 

787 

500 

341 

3,500 

1424 

818 

512 

346 

4,000 

1500 

842 

522 

350 

4,500 

1566 

863 

530 

354 

5.000 

1622 

879 

536 

357 

6,000 

1715 

906 

546 

361 

7,000 

1788 

926 

553 

364 

8,000 

1847 

942 

558 

367 

9,000 

1895 

954 

563 

368 

10,000 

1936 

964 

566 

370 

15,000 

2070 

996 

577 

375 

20,000 

2144 

1013 

583 

377 

25,000 

2191 

1023 

586 

378 

50,000 

2291 

1045 

593 

381 

100,000 

2345 

1056 

597 

383 

Note:  This  ta 

bie  should  be  used  only  when  the  auditor  is  unabi 

le  or 

unwilling  to  fix  a  maximum  occurrence  rate  to  be  expected.  This 


conservative  approach  will  result  in  a  much  larger  sample  size  than 
will  be  found  in  tables  where  an  expected  maximum  rate  is  estimated. 
Source;  Sampling  in  Auditing,  Hill, 1962.  p.  A-1  |  _ 
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